Template for test
In [1]:
from pred import Predictor
from pred import sequence_vector
from pred import chemical_vector
/Users/mark/anaconda3/lib/python3.6/site-packages/sklearn/cross_validation.py:44: DeprecationWarning: This module was deprecated in version 0.18 in favor of the model_selection module into which all the refactored classes and functions are moved. Also note that the interface of the new CV iterators are different from that of this module. This module will be removed in 0.20.
"This module will be removed in 0.20.", DeprecationWarning)
Controlling for Random Negatve vs Sans Random in Imbalanced Techniques using S, T, and Y Phosphorylation.
Included is N Phosphorylation however no benchmarks are available, yet.
Training data is from phospho.elm and benchmarks are from dbptm.
Note: SMOTEEN seems to preform best
In [2]:
par = ["pass", "ADASYN", "SMOTEENN", "random_under_sample", "ncl", "near_miss"]
benchmarks = ["Data/Benchmarks/phos_CDK1.csv", "Data/Benchmarks/phos_CK2.csv", "Data/Benchmarks/phos_MAPK1.csv", "Data/Benchmarks/phos_PKA.csv", "Data/Benchmarks/phos_PKC.csv"]
for j in benchmarks:
for i in par:
print("y", i, " ", j)
y = Predictor()
y.load_data(file="Data/Training/clean_s_filtered.csv")
y.process_data(vector_function="sequence", amino_acid="S", imbalance_function=i, random_data=0)
y.supervised_training("bagging")
y.benchmark(j, "S")
del y
print("x", i, " ", j)
x = Predictor()
x.load_data(file="Data/Training/clean_s_filtered.csv")
x.process_data(vector_function="sequence", amino_acid="S", imbalance_function=i, random_data=1)
x.supervised_training("bagging")
x.benchmark(j, "S")
del x
y pass Data/Benchmarks/phos_CDK1.csv
Loading Data
Loaded Data
Working on Data
Sample Vector [9, 19, 8, 9, 2, 10, 10, 1, 5, 17, 20, 11, 7, -1.7000000000000002, 53.67692307692308, 0.07692307692307693]
Finished working with Data
Training Data Points: 200363
Test Data Points: 50091
Starting Training
Done training
Test Results
Sensitivity: 0.15085502668554623
Specificity : 0.9437301393302371
Accuracy: 0.7984068994430137
ROC 0.547292583008
TP 1385 FP 2302 TN 38608 FN 7796
None
Cross: Validation: [ 0.79932923 0.79824719 0.79902577 0.79966061 0.79916151]
Number of data points in benchmark 1808
Benchmark Results
Sensitivity: 0.4375
Specificity : 0.9231651376146789
Accuracy: 0.9059734513274337
ROC 0.680332568807
TP 28 FP 134 TN 1610 FN 36
None
x pass Data/Benchmarks/phos_CDK1.csv
Loading Data
Loaded Data
Working on Data
Sample Vector [1, 3, 17, 11, 15, 10, 10, 15, 16, 12, 9, 12, 8, -0.6692307692307692, 63.16923076923078, 0.15384615384615385]
Finished working with Data
Training Data Points: 200363
Test Data Points: 50091
Starting Training
Done training
Test Results
Sensitivity: 0.15572882824344814
Specificity : 0.9441956090755562
Accuracy: 0.798846100097822
ROC 0.54996221866
TP 1438 FP 2280 TN 38577 FN 7796
None
Cross: Validation: [ 0.7981514 0.79778803 0.79756843 0.80015971 0.79962068]
Number of data points in benchmark 1808
Benchmark Results
Sensitivity: 0.40625
Specificity : 0.9225917431192661
Accuracy: 0.9043141592920354
ROC 0.66442087156
TP 26 FP 135 TN 1609 FN 38
None
y ADASYN Data/Benchmarks/phos_CDK1.csv
Loading Data
Loaded Data
Working on Data
Sample Vector [3, 11, 11, 10, 10, 10, 11, 1, 11, 11, 10, 8, 0, -0.9416666666666668, 174.18333333333334, 0.0]
Balancing Data
Balanced Data
Finished working with Data
Training Data Points: 331693
Test Data Points: 82924
Starting Training
Done training
Test Results
Sensitivity: 0.804688609992896
Specificity : 0.9328402221457709
Accuracy: 0.8675775408808065
ROC 0.868764416069
TP 33982 FP 2733 TN 37961 FN 8248
None
Cross: Validation: [ 0.52102502 0.9050565 0.93869011 0.93746005 0.93657972]
Number of data points in benchmark 1808
Benchmark Results
Sensitivity: 0.421875
Specificity : 0.9111238532110092
Accuracy: 0.8938053097345132
ROC 0.666499426606
TP 27 FP 155 TN 1589 FN 37
None
x ADASYN Data/Benchmarks/phos_CDK1.csv
Loading Data
Loaded Data
Working on Data
Sample Vector [15, 4, 3, 10, 15, 17, 10, 12, 19, 13, 17, 7, 10, -0.5384615384615384, 52.261538461538464, 0.15384615384615385]
Balancing Data
Balanced Data
Finished working with Data
Training Data Points: 331713
Test Data Points: 82929
Starting Training
Done training
Test Results
Sensitivity: 0.8059857730830537
Specificity : 0.9292106807511737
Accuracy: 0.8667534879234043
ROC 0.867598226917
TP 33878 FP 2895 TN 38001 FN 8155
None
Cross: Validation: [ 0.52027011 0.90617162 0.93741559 0.93657148 0.9378497 ]
Number of data points in benchmark 1808
Benchmark Results
Sensitivity: 0.390625
Specificity : 0.8990825688073395
Accuracy: 0.8810840707964602
ROC 0.644853784404
TP 25 FP 176 TN 1568 FN 39
None
y SMOTEENN Data/Benchmarks/phos_CDK1.csv
Loading Data
Loaded Data
Working on Data
Sample Vector [12, 5, 2, 10, 5, 11, 10, 11, 11, 3, 11, 7, 19, 3.416070845000482e-17, 102.29230769230772, 0.15384615384615385]
Balancing Data
Balanced Data
Finished working with Data
Training Data Points: 305683
Test Data Points: 76421
Starting Training
Done training
Test Results
Sensitivity: 0.8110877192982456
Specificity : 0.9534022943425826
Accuracy: 0.8870598395728923
ROC 0.88224500682
TP 28895 FP 1901 TN 38895 FN 6730
None
Cross: Validation: [ 0.55877889 0.93879954 0.95575823 0.955352 0.95616331]
Number of data points in benchmark 1808
Benchmark Results
Sensitivity: 0.3125
Specificity : 0.9334862385321101
Accuracy: 0.911504424778761
ROC 0.622993119266
TP 20 FP 116 TN 1628 FN 44
None
x SMOTEENN Data/Benchmarks/phos_CDK1.csv
Loading Data
Loaded Data
Working on Data
Sample Vector [1, 14, 17, 17, 17, 16, 10, 10, 9, 20, 5, 10, 2, -1.276923076923077, 151.29230769230767, 0.07692307692307693]
Balancing Data
Balanced Data
Finished working with Data
Training Data Points: 305828
Test Data Points: 76457
Starting Training
Done training
Test Results
Sensitivity: 0.810905522848332
Specificity : 0.9531713536175742
Accuracy: 0.8867991158428921
ROC 0.882038438233
TP 28925 FP 1910 TN 38877 FN 6745
None
Cross: Validation: [ 0.56382589 0.94014937 0.95746629 0.95389565 0.95656325]
Number of data points in benchmark 1808
Benchmark Results
Sensitivity: 0.34375
Specificity : 0.9329128440366973
Accuracy: 0.9120575221238938
ROC 0.638331422018
TP 22 FP 117 TN 1627 FN 42
None
y random_under_sample Data/Benchmarks/phos_CDK1.csv
Loading Data
Loaded Data
Working on Data
Sample Vector [2, 3, 7, 17, 9, 3, 10, 3, 19, 3, 18, 11, 3, -0.03846153846153853, 25.56153846153846, 0.0]
Balancing Data
Balanced Data
Finished working with Data
Training Data Points: 73804
Test Data Points: 18452
Starting Training
Done training
Test Results
Sensitivity: 0.5983097988874626
Specificity : 0.7126537785588752
Accuracy: 0.6547257749837416
ROC 0.655481788723
TP 5593 FP 2616 TN 6488 FN 3755
None
Cross: Validation: [ 0.65927813 0.65646 0.66084977 0.65235772 0.65311653]
Number of data points in benchmark 1808
Benchmark Results
Sensitivity: 0.78125
Specificity : 0.6083715596330275
Accuracy: 0.6144911504424779
ROC 0.694810779817
TP 50 FP 683 TN 1061 FN 14
None
x random_under_sample Data/Benchmarks/phos_CDK1.csv
Loading Data
Loaded Data
Working on Data
Sample Vector [4, 19, 1, 12, 10, 20, 10, 10, 9, 9, 8, 7, 15, -1.276923076923077, 78.75384615384617, 0.07692307692307693]
Balancing Data
Balanced Data
Finished working with Data
Training Data Points: 73804
Test Data Points: 18452
Starting Training
Done training
Test Results
Sensitivity: 0.5914634146341463
Specificity : 0.7161687170474517
Accuracy: 0.6529915456319099
ROC 0.653816065841
TP 5529 FP 2584 TN 6520 FN 3819
None
Cross: Validation: [ 0.65873618 0.66513115 0.65602645 0.65707317 0.66292683]
Number of data points in benchmark 1808
Benchmark Results
Sensitivity: 0.796875
Specificity : 0.6032110091743119
Accuracy: 0.610066371681416
ROC 0.700043004587
TP 51 FP 692 TN 1052 FN 13
None
y ncl Data/Benchmarks/phos_CDK1.csv
Loading Data
Loaded Data
Working on Data
Sample Vector [12, 17, 16, 6, 8, 19, 10, 1, 15, 3, 13, 12, 15, -0.2076923076923078, 32.6, 0.23076923076923078]
Balancing Data
Balanced Data
Finished working with Data
Training Data Points: 143842
Test Data Points: 35961
Starting Training
Done training
Test Results
Sensitivity: 0.2979995651228528
Specificity : 0.9255315173934163
Accuracy: 0.76502321959901
ROC 0.611765541258
TP 2741 FP 1993 TN 24770 FN 6457
None
Cross: Validation: [ 0.76744251 0.76983399 0.7670532 0.76726919 0.76782536]
Number of data points in benchmark 1808
Benchmark Results
Sensitivity: 0.578125
Specificity : 0.8526376146788991
Accuracy: 0.8429203539823009
ROC 0.715381307339
TP 37 FP 257 TN 1487 FN 27
None
x ncl Data/Benchmarks/phos_CDK1.csv
Loading Data
Loaded Data
Working on Data
Sample Vector [16, 8, 7, 17, 11, 2, 10, 2, 17, 8, 3, 8, 3, -1.369230769230769, 103.83076923076925, 0.0]
Balancing Data
Balanced Data
Finished working with Data
Training Data Points: 143697
Test Data Points: 35925
Starting Training
Done training
Test Results
Sensitivity: 0.30940971465911565
Specificity : 0.9263732565531168
Accuracy: 0.7686847599164927
ROC 0.617891485606
TP 2841 FP 1969 TN 24774 FN 6341
None
Cross: Validation: [ 0.76512178 0.76784969 0.76659708 0.76625654 0.76666759]
Number of data points in benchmark 1808
Benchmark Results
Sensitivity: 0.53125
Specificity : 0.8595183486238532
Accuracy: 0.8478982300884956
ROC 0.695384174312
TP 34 FP 245 TN 1499 FN 30
None
y near_miss Data/Benchmarks/phos_CDK1.csv
Loading Data
Loaded Data
Working on Data
Sample Vector [12, 9, 3, 5, 18, 10, 10, 16, 11, 10, 11, 19, 1, -0.6846153846153846, 64.64615384615384, 0.07692307692307693]
Balancing Data
Balanced Data
Finished working with Data
Training Data Points: 73804
Test Data Points: 18452
Starting Training
Done training
Test Results
Sensitivity: 0.5933889602053916
Specificity : 0.6703646748681898
Accuracy: 0.6313678734012573
ROC 0.631876817537
TP 5547 FP 3001 TN 6103 FN 3801
None
Cross: Validation: [ 0.52866898 0.66448082 0.65494255 0.63848238 0.62861789]
Number of data points in benchmark 1808
Benchmark Results
Sensitivity: 0.828125
Specificity : 0.48509174311926606
Accuracy: 0.49723451327433627
ROC 0.65660837156
TP 53 FP 898 TN 846 FN 11
None
x near_miss Data/Benchmarks/phos_CDK1.csv
Loading Data
Loaded Data
Working on Data
Sample Vector [15, 20, 15, 18, 18, 3, 10, 20, 9, 10, 8, 16, 17, -1.8076923076923077, 47.14615384615385, 0.15384615384615385]
Balancing Data
Balanced Data
Finished working with Data
Training Data Points: 73804
Test Data Points: 18452
Starting Training
Done training
Test Results
Sensitivity: 0.5864356011981172
Specificity : 0.6673989455184535
Accuracy: 0.626381964014741
ROC 0.626917273358
TP 5482 FP 3028 TN 6076 FN 3866
None
Cross: Validation: [ 0.53300455 0.65559289 0.65716454 0.63788618 0.62373984]
Number of data points in benchmark 1808
Benchmark Results
Sensitivity: 0.828125
Specificity : 0.4764908256880734
Accuracy: 0.4889380530973451
ROC 0.652307912844
TP 53 FP 913 TN 831 FN 11
None
y pass Data/Benchmarks/phos_CK2.csv
Loading Data
Loaded Data
Working on Data
Sample Vector [18, 16, 2, 2, 11, 10, 10, 18, 2, 10, 11, 11, 9, -1.1923076923076923, 115.66153846153846, 0.0]
Finished working with Data
Training Data Points: 200363
Test Data Points: 50091
Starting Training
Done training
Test Results
Sensitivity: 0.14637040232984574
Specificity : 0.945859872611465
Accuracy: 0.7978878441236948
ROC 0.546115137471
TP 1357 FP 2210 TN 38610 FN 7914
None
Cross: Validation: [ 0.79952887 0.79764828 0.79718912 0.8 0.7982232 ]
Number of data points in benchmark 1148
Benchmark Results
Sensitivity: 0.38271604938271603
Specificity : 0.9287722586691659
Accuracy: 0.8902439024390244
ROC 0.655744154026
TP 31 FP 76 TN 991 FN 50
None
x pass Data/Benchmarks/phos_CK2.csv
Loading Data
Loaded Data
Working on Data
Sample Vector [17, 10, 7, 9, 11, 10, 10, 11, 1, 20, 19, 19, 12, -1.646153846153846, 114.74615384615386, 0.0]
Finished working with Data
Training Data Points: 200363
Test Data Points: 50091
Starting Training
Done training
Test Results
Sensitivity: 0.1464797774448962
Specificity : 0.9470364461897165
Accuracy: 0.7976682437962908
ROC 0.546758111817
TP 1369 FP 2158 TN 38587 FN 7977
None
Cross: Validation: [ 0.79811147 0.80002396 0.79930526 0.79948093 0.79988022]
Number of data points in benchmark 1148
Benchmark Results
Sensitivity: 0.35802469135802467
Specificity : 0.9325210871602624
Accuracy: 0.89198606271777
ROC 0.645272889259
TP 29 FP 72 TN 995 FN 52
None
y ADASYN Data/Benchmarks/phos_CK2.csv
Loading Data
Loaded Data
Working on Data
Sample Vector [7, 6, 17, 17, 18, 11, 10, 19, 12, 10, 19, 9, 10, -2.1230769230769235, 90.20000000000002, 0.07692307692307693]
Balancing Data
Balanced Data
Finished working with Data
Training Data Points: 331728
Test Data Points: 82933
Starting Training
Done training
Test Results
Sensitivity: 0.804170917459489
Specificity : 0.932056688897607
Accuracy: 0.8670613627868279
ROC 0.868113803179
TP 33895 FP 2771 TN 38013 FN 8254
None
Cross: Validation: [ 0.51903344 0.90816573 0.93796122 0.93858824 0.93726185]
Number of data points in benchmark 1148
Benchmark Results
Sensitivity: 0.38271604938271603
Specificity : 0.9025304592314901
Accuracy: 0.8658536585365854
ROC 0.642623254307
TP 31 FP 104 TN 963 FN 50
None
x ADASYN Data/Benchmarks/phos_CK2.csv
Loading Data
Loaded Data
Working on Data
Sample Vector [10, 19, 11, 20, 9, 19, 10, 1, 10, 19, 10, 8, 19, -2.0692307692307694, 98.12307692307694, 0.0]
Balancing Data
Balanced Data
Finished working with Data
Training Data Points: 331712
Test Data Points: 82929
Starting Training
Done training
Test Results
Sensitivity: 0.8094703126493928
Specificity : 0.930158421142287
Accuracy: 0.8692737160703734
ROC 0.869814366896
TP 33865 FP 2870 TN 38223 FN 7971
None
Cross: Validation: [ 0.51952875 0.90782365 0.93812705 0.93710206 0.94028555]
Number of data points in benchmark 1148
Benchmark Results
Sensitivity: 0.35802469135802467
Specificity : 0.9119025304592315
Accuracy: 0.872822299651568
ROC 0.634963610909
TP 29 FP 94 TN 973 FN 52
None
y SMOTEENN Data/Benchmarks/phos_CK2.csv
Loading Data
Loaded Data
Working on Data
Sample Vector [9, 5, 15, 17, 18, 16, 10, 8, 11, 11, 11, 3, 9, -1.6923076923076925, 90.0546153846154, 0.15384615384615385]
Balancing Data
Balanced Data
Finished working with Data
Training Data Points: 305904
Test Data Points: 76476
Starting Training
Done training
Test Results
Sensitivity: 0.8092720049241788
Specificity : 0.9544115480925026
Accuracy: 0.886578795962132
ROC 0.881841776508
TP 28925 FP 1857 TN 38877 FN 6817
None
Cross: Validation: [ 0.55986767 0.93892201 0.95488781 0.95537162 0.95652174]
Number of data points in benchmark 1148
Benchmark Results
Sensitivity: 0.345679012345679
Specificity : 0.9231490159325211
Accuracy: 0.882404181184669
ROC 0.634414014139
TP 28 FP 82 TN 985 FN 53
None
x SMOTEENN Data/Benchmarks/phos_CK2.csv
Loading Data
Loaded Data
Working on Data
Sample Vector [17, 17, 12, 8, 3, 1, 10, 12, 9, 7, 10, 1, 10, -0.8384615384615386, 53.292307692307695, 0.0]
Balancing Data
Balanced Data
Finished working with Data
Training Data Points: 305813
Test Data Points: 76454
Starting Training
Done training
Test Results
Sensitivity: 0.8111703620544485
Specificity : 0.9548059964726632
Accuracy: 0.8878672142726345
ROC 0.882988179264
TP 28902 FP 1845 TN 38979 FN 6728
None
Cross: Validation: [ 0.55780524 0.94104875 0.95556747 0.95572443 0.95395864]
Number of data points in benchmark 1148
Benchmark Results
Sensitivity: 0.30864197530864196
Specificity : 0.936269915651359
Accuracy: 0.89198606271777
ROC 0.62245594548
TP 25 FP 68 TN 999 FN 56
None
y random_under_sample Data/Benchmarks/phos_CK2.csv
Loading Data
Loaded Data
Working on Data
Sample Vector [8, 2, 7, 9, 9, 8, 10, 16, 20, 8, 2, 3, 3, -1.146153846153846, 71.32307692307691, 0.0]
Balancing Data
Balanced Data
Finished working with Data
Training Data Points: 73804
Test Data Points: 18452
Starting Training
Done training
Test Results
Sensitivity: 0.6051561831407788
Specificity : 0.7139718804920914
Accuracy: 0.6588445696943421
ROC 0.659564031816
TP 5657 FP 2604 TN 6500 FN 3691
None
Cross: Validation: [ 0.65792326 0.65949491 0.65646 0.65723577 0.66439024]
Number of data points in benchmark 1148
Benchmark Results
Sensitivity: 0.654320987654321
Specificity : 0.6016869728209935
Accuracy: 0.6054006968641115
ROC 0.628003980238
TP 53 FP 425 TN 642 FN 28
None
x random_under_sample Data/Benchmarks/phos_CK2.csv
Loading Data
Loaded Data
Working on Data
Sample Vector [19, 9, 19, 2, 10, 1, 10, 9, 19, 9, 9, 5, 13, -1.3384615384615386, 78.75384615384615, 0.07692307692307693]
Balancing Data
Balanced Data
Finished working with Data
Training Data Points: 73804
Test Data Points: 18452
Starting Training
Done training
Test Results
Sensitivity: 0.6030166880616175
Specificity : 0.711335676625659
Accuracy: 0.6564600043355734
ROC 0.657176182344
TP 5637 FP 2628 TN 6476 FN 3711
None
Cross: Validation: [ 0.65900715 0.65743551 0.65564708 0.65387534 0.66070461]
Number of data points in benchmark 1148
Benchmark Results
Sensitivity: 0.7654320987654321
Specificity : 0.569821930646673
Accuracy: 0.5836236933797909
ROC 0.667627014706
TP 62 FP 459 TN 608 FN 19
None
y ncl Data/Benchmarks/phos_CK2.csv
Loading Data
Loaded Data
Working on Data
Sample Vector [15, 19, 19, 17, 10, 11, 10, 3, 1, 10, 3, 10, 1, -0.8307692307692309, 104.06153846153846, 0.07692307692307693]
Balancing Data
Balanced Data
Finished working with Data
Training Data Points: 143667
Test Data Points: 35917
Starting Training
Done training
Test Results
Sensitivity: 0.30934000217462215
Specificity : 0.9243263473053892
Accuracy: 0.7668513517275942
ROC 0.61683317474
TP 2845 FP 2022 TN 24698 FN 6352
None
Cross: Validation: [ 0.76724762 0.76637804 0.7666843 0.76898875 0.76500724]
Number of data points in benchmark 1148
Benchmark Results
Sensitivity: 0.3950617283950617
Specificity : 0.8650421743205249
Accuracy: 0.8318815331010453
ROC 0.630051951358
TP 32 FP 144 TN 923 FN 49
None
x ncl Data/Benchmarks/phos_CK2.csv
Loading Data
Loaded Data
Working on Data
Sample Vector [2, 18, 10, 10, 11, 1, 10, 17, 7, 10, 20, 10, 11, -1.415384615384616, 106.46153846153847, 0.0]
Balancing Data
Balanced Data
Finished working with Data
Training Data Points: 143700
Test Data Points: 35926
Starting Training
Done training
Test Results
Sensitivity: 0.3074158771643254
Specificity : 0.9269341509927832
Accuracy: 0.7685798585982297
ROC 0.617175014079
TP 2823 FP 1954 TN 24789 FN 6360
None
Cross: Validation: [ 0.76549017 0.76802316 0.76899738 0.7690402 0.76839996]
Number of data points in benchmark 1148
Benchmark Results
Sensitivity: 0.5185185185185185
Specificity : 0.865979381443299
Accuracy: 0.8414634146341463
ROC 0.692248949981
TP 42 FP 143 TN 924 FN 39
None
y near_miss Data/Benchmarks/phos_CK2.csv
Loading Data
Loaded Data
Working on Data
Sample Vector [8, 5, 20, 11, 10, 6, 10, 3, 13, 18, 10, 5, 3, 0.3923076923076923, 18.24615384615385, 0.23076923076923078]
Balancing Data
Balanced Data
Finished working with Data
Training Data Points: 73804
Test Data Points: 18452
Starting Training
Done training
Test Results
Sensitivity: 0.5799101412066753
Specificity : 0.6740992970123023
Accuracy: 0.626381964014741
ROC 0.627004719109
TP 5421 FP 2967 TN 6137 FN 3927
None
Cross: Validation: [ 0.52758509 0.66328853 0.65846521 0.63815718 0.62596206]
Number of data points in benchmark 1148
Benchmark Results
Sensitivity: 0.6790123456790124
Specificity : 0.44704779756326146
Accuracy: 0.4634146341463415
ROC 0.563030071621
TP 55 FP 590 TN 477 FN 26
None
x near_miss Data/Benchmarks/phos_CK2.csv
Loading Data
Loaded Data
Working on Data
Sample Vector [17, 16, 7, 8, 8, 9, 10, 5, 19, 3, 9, 9, 16, -2.3076923076923075, 133.66153846153847, 0.07692307692307693]
Balancing Data
Balanced Data
Finished working with Data
Training Data Points: 73804
Test Data Points: 18452
Starting Training
Done training
Test Results
Sensitivity: 0.5805519897304237
Specificity : 0.6754173989455184
Accuracy: 0.6273574680251464
ROC 0.627984694338
TP 5427 FP 2955 TN 6149 FN 3921
None
Cross: Validation: [ 0.52888576 0.66036202 0.65971168 0.63853659 0.62872629]
Number of data points in benchmark 1148
Benchmark Results
Sensitivity: 0.7530864197530864
Specificity : 0.4442361761949391
Accuracy: 0.46602787456445993
ROC 0.598661297974
TP 61 FP 593 TN 474 FN 20
None
y pass Data/Benchmarks/phos_MAPK1.csv
Loading Data
Loaded Data
Working on Data
Sample Vector [17, 2, 11, 20, 7, 9, 10, 10, 9, 9, 9, 15, 19, -2.253846153846154, 136.8615384615385, 0.07692307692307693]
Finished working with Data
Training Data Points: 200363
Test Data Points: 50091
Starting Training
Done training
Test Results
Sensitivity: 0.1462510897994769
Specificity : 0.9446657704998167
Accuracy: 0.7984068994430137
ROC 0.54545843015
TP 1342 FP 2264 TN 38651 FN 7834
None
Cross: Validation: [ 0.79879023 0.79816734 0.80020363 0.80045917 0.79880216]
Number of data points in benchmark 741
Benchmark Results
Sensitivity: 0.3548387096774194
Specificity : 0.923943661971831
Accuracy: 0.9001349527665317
ROC 0.639391185825
TP 11 FP 54 TN 656 FN 20
None
x pass Data/Benchmarks/phos_MAPK1.csv
Loading Data
Loaded Data
Working on Data
Sample Vector [18, 3, 13, 12, 16, 9, 10, 19, 1, 9, 6, 2, 1, -0.41538461538461546, -11.407692307692304, 0.07692307692307693]
Finished working with Data
Training Data Points: 200363
Test Data Points: 50091
Starting Training
Done training
Test Results
Sensitivity: 0.14090666666666668
Specificity : 0.9486197072403969
Accuracy: 0.7974486434688867
ROC 0.544763186954
TP 1321 FP 2092 TN 38624 FN 8054
None
Cross: Validation: [ 0.80074663 0.79922541 0.79910563 0.80061889 0.79934119]
Number of data points in benchmark 741
Benchmark Results
Sensitivity: 0.3548387096774194
Specificity : 0.9366197183098591
Accuracy: 0.9122807017543859
ROC 0.645729213994
TP 11 FP 45 TN 665 FN 20
None
y ADASYN Data/Benchmarks/phos_MAPK1.csv
Loading Data
Loaded Data
Working on Data
Sample Vector [12, 13, 19, 17, 10, 18, 3, 16, 9, 2, 10, 0, 0, -0.5000000000000001, 42.18181818181818, 0.0]
Balancing Data
Balanced Data
Finished working with Data
Training Data Points: 331710
Test Data Points: 82928
Starting Training
Done training
Test Results
Sensitivity: 0.805448672272101
Specificity : 0.9288198696908833
Accuracy: 0.8661851244453019
ROC 0.867134270981
TP 33911 FP 2906 TN 37920 FN 8191
None
Cross: Validation: [ 0.51875701 0.90734131 0.93645013 0.93643807 0.93724601]
Number of data points in benchmark 741
Benchmark Results
Sensitivity: 0.2903225806451613
Specificity : 0.9225352112676056
Accuracy: 0.8960863697705803
ROC 0.606428895956
TP 9 FP 55 TN 655 FN 22
None
x ADASYN Data/Benchmarks/phos_MAPK1.csv
Loading Data
Loaded Data
Working on Data
Sample Vector [5, 7, 7, 11, 20, 17, 10, 18, 4, 8, 17, 10, 7, -2.0692307692307694, 53.66923076923077, 0.07692307692307693]
Balancing Data
Balanced Data
Finished working with Data
Training Data Points: 331714
Test Data Points: 82929
Starting Training
Done training
Test Results
Sensitivity: 0.8022803773763808
Specificity : 0.9326510075350367
Accuracy: 0.8663314401475961
ROC 0.867465692456
TP 33845 FP 2744 TN 37999 FN 8341
None
Cross: Validation: [ 0.52034246 0.90694449 0.93827175 0.93778941 0.93758441]
Number of data points in benchmark 741
Benchmark Results
Sensitivity: 0.3870967741935484
Specificity : 0.923943661971831
Accuracy: 0.9014844804318488
ROC 0.655520218083
TP 12 FP 54 TN 656 FN 19
None
y SMOTEENN Data/Benchmarks/phos_MAPK1.csv
Loading Data
Loaded Data
Working on Data
Sample Vector [11, 14, 3, 11, 12, 3, 10, 17, 3, 3, 5, 10, 10, 1.1230769230769229, 77.50769230769232, 0.07692307692307693]
Balancing Data
Balanced Data
Finished working with Data
Training Data Points: 305781
Test Data Points: 76446
Starting Training
Done training
Test Results
Sensitivity: 0.8110099453705001
Specificity : 0.9552158229245908
Accuracy: 0.8878816419433325
ROC 0.883112884148
TP 28949 FP 1825 TN 38926 FN 6746
None
Cross: Validation: [ 0.55900166 0.94021846 0.95400615 0.95692328 0.95636078]
Number of data points in benchmark 741
Benchmark Results
Sensitivity: 0.2903225806451613
Specificity : 0.9380281690140845
Accuracy: 0.9109311740890689
ROC 0.61417537483
TP 9 FP 44 TN 666 FN 22
None
x SMOTEENN Data/Benchmarks/phos_MAPK1.csv
Loading Data
Loaded Data
Working on Data
Sample Vector [20, 12, 12, 1, 3, 10, 10, 7, 20, 3, 17, 2, 20, 0.40769230769230747, 32.330769230769235, 0.0]
Balancing Data
Balanced Data
Finished working with Data
Training Data Points: 305800
Test Data Points: 76451
Starting Training
Done training
Test Results
Sensitivity: 0.8112139052425007
Specificity : 0.9572840293273829
Accuracy: 0.8891316006330853
ROC 0.884248967285
TP 28936 FP 1742 TN 39039 FN 6734
None
Cross: Validation: [ 0.56140534 0.94 0.95657292 0.95662525 0.95546109]
Number of data points in benchmark 741
Benchmark Results
Sensitivity: 0.5161290322580645
Specificity : 0.9408450704225352
Accuracy: 0.9230769230769231
ROC 0.72848705134
TP 16 FP 42 TN 668 FN 15
None
y random_under_sample Data/Benchmarks/phos_MAPK1.csv
Loading Data
Loaded Data
Working on Data
Sample Vector [10, 13, 2, 19, 2, 9, 10, 3, 4, 19, 10, 10, 1, -0.02307692307692314, 47.14615384615385, 0.0]
Balancing Data
Balanced Data
Finished working with Data
Training Data Points: 73804
Test Data Points: 18452
Starting Training
Done training
Test Results
Sensitivity: 0.603872486093282
Specificity : 0.7251757469244289
Accuracy: 0.6637220897463689
ROC 0.664524116509
TP 5645 FP 2502 TN 6602 FN 3703
None
Cross: Validation: [ 0.65900715 0.65499675 0.65862779 0.65756098 0.65940379]
Number of data points in benchmark 741
Benchmark Results
Sensitivity: 0.8064516129032258
Specificity : 0.6126760563380281
Accuracy: 0.6207827260458839
ROC 0.709563834621
TP 25 FP 275 TN 435 FN 6
None
x random_under_sample Data/Benchmarks/phos_MAPK1.csv
Loading Data
Loaded Data
Working on Data
Sample Vector [10, 1, 12, 10, 1, 9, 10, 9, 8, 7, 3, 17, 9, -1.353846153846154, 56.16153846153847, 0.0]
Balancing Data
Balanced Data
Finished working with Data
Training Data Points: 73804
Test Data Points: 18452
Starting Training
Done training
Test Results
Sensitivity: 0.6065468549422336
Specificity : 0.7012302284710018
Accuracy: 0.6532625189681336
ROC 0.653888541707
TP 5670 FP 2720 TN 6384 FN 3678
None
Cross: Validation: [ 0.65646 0.65819423 0.65906135 0.65252033 0.65306233]
Number of data points in benchmark 741
Benchmark Results
Sensitivity: 0.8709677419354839
Specificity : 0.5985915492957746
Accuracy: 0.6099865047233468
ROC 0.734779645616
TP 27 FP 285 TN 425 FN 4
None
y ncl Data/Benchmarks/phos_MAPK1.csv
Loading Data
Loaded Data
Working on Data
Sample Vector [3, 10, 1, 15, 7, 1, 10, 10, 20, 11, 20, 7, 1, -0.9153846153846155, 4.453846153846155, 0.07692307692307693]
Balancing Data
Balanced Data
Finished working with Data
Training Data Points: 143812
Test Data Points: 35953
Starting Training
Done training
Test Results
Sensitivity: 0.31593824744509674
Specificity : 0.9276770697065969
Accuracy: 0.7711734764831863
ROC 0.621807658576
TP 2906 FP 1935 TN 24820 FN 6292
None
Cross: Validation: [ 0.76820382 0.76514435 0.76841988 0.77091678 0.76616044]
Number of data points in benchmark 741
Benchmark Results
Sensitivity: 0.6774193548387096
Specificity : 0.8591549295774648
Accuracy: 0.8515519568151148
ROC 0.768287142208
TP 21 FP 100 TN 610 FN 10
None
x ncl Data/Benchmarks/phos_MAPK1.csv
Loading Data
Loaded Data
Working on Data
Sample Vector [15, 2, 1, 7, 5, 1, 10, 11, 16, 2, 19, 8, 3, -0.6461538461538461, 48.184615384615384, 0.15384615384615385]
Balancing Data
Balanced Data
Finished working with Data
Training Data Points: 143720
Test Data Points: 35930
Starting Training
Done training
Test Results
Sensitivity: 0.3166123778501629
Specificity : 0.9250374251497006
Accuracy: 0.7690787642638464
ROC 0.6208249015
TP 2916 FP 2003 TN 24717 FN 6294
None
Cross: Validation: [ 0.7677493 0.76735966 0.76554411 0.76804253 0.76993515]
Number of data points in benchmark 741
Benchmark Results
Sensitivity: 0.6451612903225806
Specificity : 0.8718309859154929
Accuracy: 0.8623481781376519
ROC 0.758496138119
TP 20 FP 91 TN 619 FN 11
None
y near_miss Data/Benchmarks/phos_MAPK1.csv
Loading Data
Loaded Data
Working on Data
Sample Vector [11, 10, 2, 2, 11, 17, 10, 19, 2, 14, 3, 3, 1, 0.17692307692307693, 115.47692307692309, 0.0]
Balancing Data
Balanced Data
Finished working with Data
Training Data Points: 73804
Test Data Points: 18452
Starting Training
Done training
Test Results
Sensitivity: 0.5838682071031237
Specificity : 0.6736599297012302
Accuracy: 0.6281703880338174
ROC 0.628764068402
TP 5458 FP 2971 TN 6133 FN 3890
None
Cross: Validation: [ 0.53273358 0.65662259 0.6595491 0.63853659 0.62655827]
Number of data points in benchmark 741
Benchmark Results
Sensitivity: 0.7419354838709677
Specificity : 0.4535211267605634
Accuracy: 0.46558704453441296
ROC 0.597728305316
TP 23 FP 388 TN 322 FN 8
None
x near_miss Data/Benchmarks/phos_MAPK1.csv
Loading Data
Loaded Data
Working on Data
Sample Vector [20, 11, 19, 13, 12, 2, 10, 18, 16, 10, 5, 19, 3, -0.038461538461538464, 6.392307692307692, 0.07692307692307693]
Balancing Data
Balanced Data
Finished working with Data
Training Data Points: 73804
Test Data Points: 18452
Starting Training
Done training
Test Results
Sensitivity: 0.5870774497218656
Specificity : 0.6751977152899824
Accuracy: 0.6305549533925862
ROC 0.631137582506
TP 5488 FP 2957 TN 6147 FN 3860
None
Cross: Validation: [ 0.52807284 0.6605246 0.65434641 0.63734417 0.62178862]
Number of data points in benchmark 741
Benchmark Results
Sensitivity: 0.8387096774193549
Specificity : 0.46901408450704224
Accuracy: 0.4844804318488529
ROC 0.653861880963
TP 26 FP 377 TN 333 FN 5
None
y pass Data/Benchmarks/phos_PKA.csv
Loading Data
Loaded Data
Working on Data
Sample Vector [7, 10, 8, 19, 5, 10, 10, 5, 19, 19, 20, 0, 0, -1.4, 52.31818181818183, 0.18181818181818182]
Finished working with Data
Training Data Points: 200363
Test Data Points: 50091
Starting Training
Done training
Test Results
Sensitivity: 0.1452213941253836
Specificity : 0.9472746356823785
Accuracy: 0.8011818490347568
ROC 0.546248014904
TP 1325 FP 2160 TN 38807 FN 7799
None
Cross: Validation: [ 0.79918949 0.79692959 0.79926534 0.80085845 0.79824316]
Number of data points in benchmark 2971
Benchmark Results
Sensitivity: 0.3148148148148148
Specificity : 0.9384122463510146
Accuracy: 0.9044092898014137
ROC 0.626613530583
TP 51 FP 173 TN 2636 FN 111
None
x pass Data/Benchmarks/phos_PKA.csv
Loading Data
Loaded Data
Working on Data
Sample Vector [7, 10, 12, 5, 11, 3, 10, 11, 12, 9, 12, 3, 6, 0.7615384615384616, 90.87692307692309, 0.15384615384615385]
Finished working with Data
Training Data Points: 200363
Test Data Points: 50091
Starting Training
Done training
Test Results
Sensitivity: 0.15782000873743993
Specificity : 0.9438133626481007
Accuracy: 0.8001437383961191
ROC 0.550816685693
TP 1445 FP 2300 TN 38635 FN 7711
None
Cross: Validation: [ 0.80066677 0.79828712 0.79806752 0.79696546 0.79856259]
Number of data points in benchmark 2971
Benchmark Results
Sensitivity: 0.25308641975308643
Specificity : 0.930936276254895
Accuracy: 0.8939750925614272
ROC 0.592011348004
TP 41 FP 194 TN 2615 FN 121
None
y ADASYN Data/Benchmarks/phos_PKA.csv
Loading Data
Loaded Data
Working on Data
Sample Vector [5, 10, 12, 1, 17, 8, 10, 10, 2, 11, 2, 0, 0, -0.1636363636363636, 111.35454545454546, 0.09090909090909091]
Balancing Data
Balanced Data
Finished working with Data
Training Data Points: 331717
Test Data Points: 82930
Starting Training
Done training
Test Results
Sensitivity: 0.8057640560827078
Specificity : 0.9295927423331033
Accuracy: 0.8663330519715423
ROC 0.867678399208
TP 34137 FP 2856 TN 37708 FN 8229
None
Cross: Validation: [ 0.51920271 0.90778859 0.93660842 0.93775398 0.93721135]
Number of data points in benchmark 2971
Benchmark Results
Sensitivity: 0.3148148148148148
Specificity : 0.9216803132787469
Accuracy: 0.8885897004375631
ROC 0.618247564047
TP 51 FP 220 TN 2589 FN 111
None
x ADASYN Data/Benchmarks/phos_PKA.csv
Loading Data
Loaded Data
Working on Data
Sample Vector [17, 10, 5, 10, 2, 14, 10, 12, 1, 3, 1, 17, 10, 0.16153846153846152, 104.1, 0.07692307692307693]
Balancing Data
Balanced Data
Finished working with Data
Training Data Points: 331715
Test Data Points: 82929
Starting Training
Done training
Test Results
Sensitivity: 0.8062212309590471
Specificity : 0.9309025819581689
Accuracy: 0.8675372909356196
ROC 0.868561906459
TP 33979 FP 2818 TN 37965 FN 8167
None
Cross: Validation: [ 0.51861811 0.90749919 0.93740429 0.93909174 0.93717442]
Number of data points in benchmark 2971
Benchmark Results
Sensitivity: 0.3148148148148148
Specificity : 0.9184763260946956
Accuracy: 0.8855604173678896
ROC 0.616645570455
TP 51 FP 229 TN 2580 FN 111
None
y SMOTEENN Data/Benchmarks/phos_PKA.csv
Loading Data
Loaded Data
Working on Data
Sample Vector [19, 19, 20, 4, 10, 12, 13, 10, 1, 4, 10, 0, 0, 0.18181818181818177, 75.31818181818181, 0.0]
Balancing Data
Balanced Data
Finished working with Data
Training Data Points: 305648
Test Data Points: 76413
Starting Training
Done training
Test Results
Sensitivity: 0.8099400459460974
Specificity : 0.9561384120435178
Accuracy: 0.8878463088741444
ROC 0.883039228995
TP 28910 FP 1786 TN 38933 FN 6784
None
Cross: Validation: [ 0.55991782 0.94091242 0.95571376 0.95529498 0.95550437]
Number of data points in benchmark 2971
Benchmark Results
Sensitivity: 0.2037037037037037
Specificity : 0.9366322534709861
Accuracy: 0.8966677886233592
ROC 0.570167978587
TP 33 FP 178 TN 2631 FN 129
None
x SMOTEENN Data/Benchmarks/phos_PKA.csv
Loading Data
Loaded Data
Working on Data
Sample Vector [12, 16, 10, 11, 11, 2, 10, 3, 12, 11, 17, 13, 0, 0.3666666666666667, 87.65, 0.0]
Balancing Data
Balanced Data
Finished working with Data
Training Data Points: 232136
Test Data Points: 58035
Starting Training
Done training
Test Results
Sensitivity: 0.8922850484484239
Specificity : 0.8254198031268095
Accuracy: 0.8723873524597225
ROC 0.858852425788
TP 36374 FP 3015 TN 14255 FN 4391
None
Cross: Validation: [ 0.48860171 0.89921425 0.93010994 0.92895544 0.92874866]
Number of data points in benchmark 2971
Benchmark Results
Sensitivity: 0.6172839506172839
Specificity : 0.7230331078675686
Accuracy: 0.717266913497139
ROC 0.670158529242
TP 100 FP 778 TN 2031 FN 62
None
y random_under_sample Data/Benchmarks/phos_PKA.csv
Loading Data
Loaded Data
Working on Data
Sample Vector [17, 10, 20, 11, 10, 10, 10, 11, 10, 3, 17, 7, 17, -1.6538461538461537, 175.98461538461538, 0.0]
Balancing Data
Balanced Data
Finished working with Data
Training Data Points: 73804
Test Data Points: 18452
Starting Training
Done training
Test Results
Sensitivity: 0.6013050919982884
Specificity : 0.711225834797891
Accuracy: 0.6555386949924128
ROC 0.656265463398
TP 5621 FP 2629 TN 6475 FN 3727
None
Cross: Validation: [ 0.65944071 0.65992846 0.65868199 0.65485095 0.65869919]
Number of data points in benchmark 2971
Benchmark Results
Sensitivity: 0.7407407407407407
Specificity : 0.6290494838020648
Accuracy: 0.6351396836082127
ROC 0.684895112271
TP 120 FP 1042 TN 1767 FN 42
None
x random_under_sample Data/Benchmarks/phos_PKA.csv
Loading Data
Loaded Data
Working on Data
Sample Vector [1, 2, 7, 13, 19, 2, 10, 7, 18, 9, 9, 19, 9, -1.6846153846153844, 36.06153846153847, 0.0]
Balancing Data
Balanced Data
Finished working with Data
Training Data Points: 73804
Test Data Points: 18452
Starting Training
Done training
Test Results
Sensitivity: 0.5883611467693625
Specificity : 0.7147407732864675
Accuracy: 0.6507153696076307
ROC 0.651550960028
TP 5500 FP 2597 TN 6507 FN 3848
None
Cross: Validation: [ 0.66404726 0.65369608 0.65402124 0.65279133 0.65598916]
Number of data points in benchmark 2971
Benchmark Results
Sensitivity: 0.691358024691358
Specificity : 0.6379494482022072
Accuracy: 0.6408616627398183
ROC 0.664653736447
TP 112 FP 1017 TN 1792 FN 50
None
y ncl Data/Benchmarks/phos_PKA.csv
Loading Data
Loaded Data
Working on Data
Sample Vector [10, 2, 1, 11, 10, 2, 10, 11, 12, 11, 10, 2, 11, -0.030769230769230795, 117.10769230769232, 0.0]
Balancing Data
Balanced Data
Finished working with Data
Training Data Points: 143673
Test Data Points: 35919
Starting Training
Done training
Test Results
Sensitivity: 0.3138344226579521
Specificity : 0.9249410972736453
Accuracy: 0.7687574821125309
ROC 0.619387759966
TP 2881 FP 2007 TN 24732 FN 6299
None
Cross: Validation: [ 0.7646371 0.76959269 0.76811715 0.76671864 0.76896734]
Number of data points in benchmark 2971
Benchmark Results
Sensitivity: 0.4691358024691358
Specificity : 0.8846564613741545
Accuracy: 0.8619993268259846
ROC 0.676896131922
TP 76 FP 324 TN 2485 FN 86
None
x ncl Data/Benchmarks/phos_PKA.csv
Loading Data
Loaded Data
Working on Data
Sample Vector [20, 10, 18, 10, 18, 10, 10, 1, 5, 20, 18, 10, 2, -0.9000000000000002, 12.484615384615388, 0.07692307692307693]
Balancing Data
Balanced Data
Finished working with Data
Training Data Points: 143784
Test Data Points: 35946
Starting Training
Done training
Test Results
Sensitivity: 0.3104759834818518
Specificity : 0.923459467544122
Accuracy: 0.7665386969342903
ROC 0.616967725513
TP 2857 FP 2047 TN 24697 FN 6345
None
Cross: Validation: [ 0.76801958 0.76824213 0.76826351 0.76873 0.76603144]
Number of data points in benchmark 2971
Benchmark Results
Sensitivity: 0.46296296296296297
Specificity : 0.8764684941260235
Accuracy: 0.8539212386401885
ROC 0.669715728544
TP 75 FP 347 TN 2462 FN 87
None
y near_miss Data/Benchmarks/phos_PKA.csv
Loading Data
Loaded Data
Working on Data
Sample Vector [13, 17, 9, 8, 2, 10, 10, 15, 10, 3, 8, 8, 7, -1.2307692307692308, 78.75384615384617, 0.07692307692307693]
Balancing Data
Balanced Data
Finished working with Data
Training Data Points: 73804
Test Data Points: 18452
Starting Training
Done training
Test Results
Sensitivity: 0.5875053487376979
Specificity : 0.672231985940246
Accuracy: 0.6293084760459571
ROC 0.629868667339
TP 5492 FP 2984 TN 6120 FN 3856
None
Cross: Validation: [ 0.52774767 0.67038803 0.65776068 0.640271 0.62552846]
Number of data points in benchmark 2971
Benchmark Results
Sensitivity: 0.7777777777777778
Specificity : 0.4734781060875756
Accuracy: 0.49007068327162573
ROC 0.625627941933
TP 126 FP 1479 TN 1330 FN 36
None
x near_miss Data/Benchmarks/phos_PKA.csv
Loading Data
Loaded Data
Working on Data
Sample Vector [9, 9, 9, 13, 20, 10, 11, 10, 19, 1, 10, 3, 0, -0.9, 132.21666666666667, 0.0]
Balancing Data
Balanced Data
Finished working with Data
Training Data Points: 73804
Test Data Points: 18452
Starting Training
Done training
Test Results
Sensitivity: 0.5876123234916559
Specificity : 0.6747583479789103
Accuracy: 0.6306091480598309
ROC 0.631185335735
TP 5493 FP 2961 TN 6143 FN 3855
None
Cross: Validation: [ 0.52996965 0.65770648 0.6596033 0.64189702 0.62303523]
Number of data points in benchmark 2971
Benchmark Results
Sensitivity: 0.8024691358024691
Specificity : 0.47561409754360984
Accuracy: 0.49343655334904074
ROC 0.639041616673
TP 130 FP 1473 TN 1336 FN 32
None
y pass Data/Benchmarks/phos_PKC.csv
Loading Data
Loaded Data
Working on Data
Sample Vector [12, 2, 12, 3, 1, 11, 10, 11, 10, 10, 12, 12, 7, 0.9615384615384615, 85.26153846153848, 0.0]
Finished working with Data
Training Data Points: 200363
Test Data Points: 50091
Starting Training
Done training
Test Results
Sensitivity: 0.15263215980576095
Specificity : 0.9461857177674872
Accuracy: 0.8026391966620751
ROC 0.549408938787
TP 1383 FP 2208 TN 38822 FN 7678
None
Cross: Validation: [ 0.79932923 0.7992853 0.80024356 0.8 0.80025953]
Number of data points in benchmark 1454
Benchmark Results
Sensitivity: 0.25
Specificity : 0.920899854862119
Accuracy: 0.8858321870701513
ROC 0.585449927431
TP 19 FP 109 TN 1269 FN 57
None
x pass Data/Benchmarks/phos_PKC.csv
Loading Data
Loaded Data
Working on Data
Sample Vector [10, 7, 10, 8, 17, 15, 10, 10, 12, 15, 0, 0, 0, -1.3500000000000003, 32.440000000000005, 0.2]
Finished working with Data
Training Data Points: 200363
Test Data Points: 50091
Starting Training
Done training
Test Results
Sensitivity: 0.149500590699173
Specificity : 0.9464443354585581
Accuracy: 0.7983070811123755
ROC 0.547972463079
TP 1392 FP 2184 TN 38596 FN 7919
None
Cross: Validation: [ 0.79741276 0.79918548 0.79870635 0.79996007 0.79746456]
Number of data points in benchmark 1454
Benchmark Results
Sensitivity: 0.23684210526315788
Specificity : 0.9339622641509434
Accuracy: 0.8975240715268226
ROC 0.585402184707
TP 18 FP 91 TN 1287 FN 58
None
y ADASYN Data/Benchmarks/phos_PKC.csv
Loading Data
Loaded Data
Working on Data
Sample Vector [2, 9, 5, 2, 10, 14, 10, 5, 2, 11, 17, 10, 2, 0.2538461538461539, 77.12307692307692, 0.15384615384615385]
Balancing Data
Balanced Data
Finished working with Data
Training Data Points: 331703
Test Data Points: 82926
Starting Training
Done training
Test Results
Sensitivity: 0.8045979732929255
Specificity : 0.9291226345539445
Accuracy: 0.8656995393483347
ROC 0.866860303923
TP 33983 FP 2884 TN 37806 FN 8253
None
Cross: Validation: [ 0.51923981 0.90794202 0.93741408 0.9364486 0.93626771]
Number of data points in benchmark 1454
Benchmark Results
Sensitivity: 0.25
Specificity : 0.9172714078374455
Accuracy: 0.8823933975240715
ROC 0.583635703919
TP 19 FP 114 TN 1264 FN 57
None
x ADASYN Data/Benchmarks/phos_PKC.csv
Loading Data
Loaded Data
Working on Data
Sample Vector [10, 1, 7, 9, 10, 10, 10, 11, 12, 2, 3, 7, 1, -0.5461538461538464, 82.70000000000002, 0.0]
Balancing Data
Balanced Data
Finished working with Data
Training Data Points: 331677
Test Data Points: 82920
Starting Training
Done training
Test Results
Sensitivity: 0.8024656167604931
Specificity : 0.9309179098993166
Accuracy: 0.8657018813314038
ROC 0.86669176333
TP 33783 FP 2820 TN 38001 FN 8316
None
Cross: Validation: [ 0.5208572 0.90958646 0.93749322 0.93862685 0.93755352]
Number of data points in benchmark 1454
Benchmark Results
Sensitivity: 0.25
Specificity : 0.9274310595065312
Accuracy: 0.8920220082530949
ROC 0.588715529753
TP 19 FP 100 TN 1278 FN 57
None
y SMOTEENN Data/Benchmarks/phos_PKC.csv
Loading Data
Loaded Data
Working on Data
Sample Vector [7, 19, 11, 19, 10, 10, 10, 7, 12, 3, 9, 3, 3, -0.5153846153846154, 41.346153846153854, 0.0]
Balancing Data
Balanced Data
Finished working with Data
Training Data Points: 305788
Test Data Points: 76448
Starting Training
Done training
Test Results
Sensitivity: 0.8102729181245626
Specificity : 0.9547430199150357
Accuracy: 0.8872305357890331
ROC 0.88250796902
TP 28947 FP 1843 TN 38880 FN 6778
None
Cross: Validation: [ 0.56129657 0.93727681 0.95511923 0.95463524 0.9555509 ]
Number of data points in benchmark 1454
Benchmark Results
Sensitivity: 0.25
Specificity : 0.9339622641509434
Accuracy: 0.8982118294360385
ROC 0.591981132075
TP 19 FP 91 TN 1287 FN 57
None
x SMOTEENN Data/Benchmarks/phos_PKC.csv
Loading Data
Loaded Data
Working on Data
Sample Vector [1, 7, 10, 10, 12, 13, 10, 3, 13, 9, 17, 5, 0, 0.42500000000000004, 54.75833333333334, 0.08333333333333333]
Balancing Data
Balanced Data
Finished working with Data
Training Data Points: 305869
Test Data Points: 76468
Starting Training
Done training
Test Results
Sensitivity: 0.8144539973202323
Specificity : 0.9521208542466293
Accuracy: 0.8876261965789611
ROC 0.883287425783
TP 29177 FP 1946 TN 38698 FN 6647
None
Cross: Validation: [ 0.56149551 0.93826095 0.95636026 0.95585024 0.95515713]
Number of data points in benchmark 1454
Benchmark Results
Sensitivity: 0.13157894736842105
Specificity : 0.9317851959361393
Accuracy: 0.889958734525447
ROC 0.531682071652
TP 10 FP 94 TN 1284 FN 66
None
y random_under_sample Data/Benchmarks/phos_PKC.csv
Loading Data
Loaded Data
Working on Data
Sample Vector [3, 8, 10, 10, 2, 18, 10, 2, 20, 20, 20, 10, 15, -0.4769230769230769, 82.92307692307692, 0.07692307692307693]
Balancing Data
Balanced Data
Finished working with Data
Training Data Points: 73804
Test Data Points: 18452
Starting Training
Done training
Test Results
Sensitivity: 0.5892169448010269
Specificity : 0.7145210896309314
Accuracy: 0.651040537611099
ROC 0.651869017216
TP 5508 FP 2599 TN 6505 FN 3840
None
Cross: Validation: [ 0.66448082 0.65808584 0.65429222 0.65696477 0.65544715]
Number of data points in benchmark 1454
Benchmark Results
Sensitivity: 0.4868421052631579
Specificity : 0.6059506531204645
Accuracy: 0.5997248968363136
ROC 0.546396379192
TP 37 FP 543 TN 835 FN 39
None
x random_under_sample Data/Benchmarks/phos_PKC.csv
Loading Data
Loaded Data
Working on Data
Sample Vector [9, 11, 8, 11, 6, 10, 10, 15, 14, 7, 17, 7, 10, -1.8923076923076925, 91.35384615384615, 0.15384615384615385]
Balancing Data
Balanced Data
Finished working with Data
Training Data Points: 73804
Test Data Points: 18452
Starting Training
Done training
Test Results
Sensitivity: 0.6050492083868207
Specificity : 0.7171572934973638
Accuracy: 0.6603620203771949
ROC 0.661103250942
TP 5656 FP 2575 TN 6529 FN 3692
None
Cross: Validation: [ 0.6585194 0.65472577 0.66095816 0.65138211 0.65479675]
Number of data points in benchmark 1454
Benchmark Results
Sensitivity: 0.5789473684210527
Specificity : 0.6219158200290276
Accuracy: 0.6196698762035764
ROC 0.600431594225
TP 44 FP 521 TN 857 FN 32
None
y ncl Data/Benchmarks/phos_PKC.csv
Loading Data
Loaded Data
Working on Data
Sample Vector [12, 13, 17, 20, 1, 5, 10, 20, 3, 7, 1, 8, 3, 0.32307692307692304, -10.361538461538462, 0.07692307692307693]
Balancing Data
Balanced Data
Finished working with Data
Training Data Points: 143686
Test Data Points: 35922
Starting Training
Done training
Test Results
Sensitivity: 0.3128814298169137
Specificity : 0.9280266208031107
Accuracy: 0.7708924892823339
ROC 0.62045402531
TP 2871 FP 1925 TN 24821 FN 6305
None
Cross: Validation: [ 0.76705083 0.7666611 0.76824787 0.77032933 0.76971688]
Number of data points in benchmark 1454
Benchmark Results
Sensitivity: 0.3026315789473684
Specificity : 0.8693759071117562
Accuracy: 0.8397524071526823
ROC 0.58600374303
TP 23 FP 180 TN 1198 FN 53
None
x ncl Data/Benchmarks/phos_PKC.csv
Loading Data
Loaded Data
Working on Data
Sample Vector [11, 2, 10, 16, 17, 16, 10, 16, 17, 16, 17, 10, 19, -2.4615384615384617, 87.47692307692309, 0.0]
Balancing Data
Balanced Data
Finished working with Data
Training Data Points: 143758
Test Data Points: 35940
Starting Training
Done training
Test Results
Sensitivity: 0.2956502779897525
Specificity : 0.9283072439944708
Accuracy: 0.766833611574847
ROC 0.611978760992
TP 2712 FP 1919 TN 24848 FN 6461
None
Cross: Validation: [ 0.77156372 0.76480245 0.76800223 0.76944267 0.76785665]
Number of data points in benchmark 1454
Benchmark Results
Sensitivity: 0.2236842105263158
Specificity : 0.8766328011611031
Accuracy: 0.842503438789546
ROC 0.550158505844
TP 17 FP 170 TN 1208 FN 59
None
y near_miss Data/Benchmarks/phos_PKC.csv
Loading Data
Loaded Data
Working on Data
Sample Vector [2, 11, 9, 19, 10, 2, 10, 11, 12, 10, 8, 8, 17, -1.2538461538461536, 145.2923076923077, 0.0]
Balancing Data
Balanced Data
Finished working with Data
Training Data Points: 73804
Test Data Points: 18452
Starting Training
Done training
Test Results
Sensitivity: 0.5859007274283269
Specificity : 0.6734402460456942
Accuracy: 0.6290916973769781
ROC 0.629670486737
TP 5477 FP 2973 TN 6131 FN 3871
None
Cross: Validation: [ 0.52574247 0.66269239 0.65792326 0.63945799 0.62775068]
Number of data points in benchmark 1454
Benchmark Results
Sensitivity: 0.7105263157894737
Specificity : 0.46879535558780844
Accuracy: 0.4814305364511692
ROC 0.589660835689
TP 54 FP 732 TN 646 FN 22
None
x near_miss Data/Benchmarks/phos_PKC.csv
Loading Data
Loaded Data
Working on Data
Sample Vector [19, 1, 19, 20, 3, 19, 10, 19, 9, 5, 14, 13, 3, -0.1538461538461539, 27.300000000000004, 0.07692307692307693]
Balancing Data
Balanced Data
Finished working with Data
Training Data Points: 73804
Test Data Points: 18452
Starting Training
Done training
Test Results
Sensitivity: 0.5862216516902011
Specificity : 0.671902460456942
Accuracy: 0.6284955560372859
ROC 0.629062056074
TP 5480 FP 2987 TN 6117 FN 3868
None
Cross: Validation: [ 0.52975287 0.66193367 0.65608064 0.63647696 0.62731707]
Number of data points in benchmark 1454
Benchmark Results
Sensitivity: 0.6973684210526315
Specificity : 0.4593613933236575
Accuracy: 0.4718019257221458
ROC 0.578364907188
TP 53 FP 745 TN 633 FN 23
None
Y Phosphorylation
In [3]:
par = ["pass", "ADASYN", "SMOTEENN", "random_under_sample", "ncl", "near_miss"]
benchmarks = ["Data/Benchmarks/phos_CDK1.csv", "Data/Benchmarks/phos_CK2.csv", "Data/Benchmarks/phos_MAPK1.csv", "Data/Benchmarks/phos_PKA.csv", "Data/Benchmarks/phos_PKC.csv"]
for j in benchmarks:
for i in par:
try:
print("y", i, " ", j)
y = Predictor()
y.load_data(file="Data/Training/clean_Y_filtered.csv")
y.process_data(vector_function="sequence", amino_acid="Y", imbalance_function=i, random_data=0)
y.supervised_training("bagging")
y.benchmark(j, "Y")
del y
print("x", i, " ", j)
x = Predictor()
x.load_data(file="Data/Training/clean_Y_filtered.csv")
x.process_data(vector_function="sequence", amino_acid="Y", imbalance_function=i, random_data=1)
x.supervised_training("bagging")
x.benchmark(j, "Y")
del x
except:
print("Benchmark not relevant")
y pass Data/Benchmarks/phos_CDK1.csv
Loading Data
Loaded Data
Working on Data
Sample Vector [19, 11, 1, 12, 13, 20, 15, 19, 3, 11, 20, 11, 11, -0.3076923076923077, 57.07692307692308, 0.07692307692307693]
Finished working with Data
Training Data Points: 10099
Test Data Points: 2525
Starting Training
Done training
Test Results
Sensitivity: 0.06474820143884892
Specificity : 0.998742665549036
Accuracy: 0.9473267326732673
ROC 0.531745433494
TP 9 FP 3 TN 2383 FN 130
None
Cross: Validation: [ 0.95447348 0.95722772 0.95287129 0.95364501 0.9544374 ]
Number of data points in benchmark 0
Benchmark not relevant
y ADASYN Data/Benchmarks/phos_CDK1.csv
Loading Data
/Users/mark/anaconda3/lib/python3.6/site-packages/sklearn/utils/validation.py:395: DeprecationWarning: Passing 1d arrays as data is deprecated in 0.17 and will raise ValueError in 0.19. Reshape your data either using X.reshape(-1, 1) if your data has a single feature or X.reshape(1, -1) if it contains a single sample.
DeprecationWarning)
Loaded Data
Working on Data
Sample Vector [2, 9, 7, 3, 4, 19, 15, 2, 13, 2, 10, 8, 13, 0.2769230769230769, 35.73076923076923, 0.07692307692307693]
Balancing Data
Balanced Data
Finished working with Data
Training Data Points: 19360
Test Data Points: 4840
Starting Training
Done training
Test Results
Sensitivity: 0.9458107544810338
Specificity : 0.9938549774682507
Accuracy: 0.9700413223140496
ROC 0.969832865975
TP 2269 FP 15 TN 2426 FN 130
None
Cross: Validation: [ 0.86180541 0.97025408 0.97809917 0.97520149 0.97706138]
Number of data points in benchmark 0
Benchmark not relevant
y SMOTEENN Data/Benchmarks/phos_CDK1.csv
Loading Data
/Users/mark/anaconda3/lib/python3.6/site-packages/sklearn/utils/validation.py:395: DeprecationWarning: Passing 1d arrays as data is deprecated in 0.17 and will raise ValueError in 0.19. Reshape your data either using X.reshape(-1, 1) if your data has a single feature or X.reshape(1, -1) if it contains a single sample.
DeprecationWarning)
Loaded Data
Working on Data
Sample Vector [12, 12, 1, 10, 8, 9, 15, 10, 2, 10, 1, 10, 11, -0.2846153846153846, 66.13076923076923, 0.07692307692307693]
Balancing Data
Balanced Data
Finished working with Data
Training Data Points: 19231
Test Data Points: 4808
Starting Training
Done training
Test Results
Sensitivity: 0.9485640805829404
Specificity : 0.9951515151515151
Accuracy: 0.9725457570715474
ROC 0.971857797867
TP 2213 FP 12 TN 2463 FN 120
None
Cross: Validation: [ 0.89062175 0.98960067 0.98710483 0.98647805 0.98398169]
Number of data points in benchmark 0
Benchmark not relevant
y random_under_sample Data/Benchmarks/phos_CDK1.csv
Loading Data
/Users/mark/anaconda3/lib/python3.6/site-packages/sklearn/utils/validation.py:395: DeprecationWarning: Passing 1d arrays as data is deprecated in 0.17 and will raise ValueError in 0.19. Reshape your data either using X.reshape(-1, 1) if your data has a single feature or X.reshape(1, -1) if it contains a single sample.
DeprecationWarning)
Loaded Data
Working on Data
Sample Vector [9, 8, 20, 2, 8, 18, 15, 7, 8, 17, 14, 20, 10, -1.9307692307692308, 67.3076923076923, 0.07692307692307693]
Balancing Data
Balanced Data
Finished working with Data
Training Data Points: 945
Test Data Points: 237
Starting Training
Done training
Test Results
Sensitivity: 0.5169491525423728
Specificity : 0.6974789915966386
Accuracy: 0.6075949367088608
ROC 0.60721407207
TP 61 FP 36 TN 83 FN 57
None
Cross: Validation: [ 0.61344538 0.62711864 0.53813559 0.54661017 0.61864407]
Number of data points in benchmark 0
Benchmark not relevant
y ncl Data/Benchmarks/phos_CDK1.csv
Loading Data
/Users/mark/anaconda3/lib/python3.6/site-packages/sklearn/utils/validation.py:395: DeprecationWarning: Passing 1d arrays as data is deprecated in 0.17 and will raise ValueError in 0.19. Reshape your data either using X.reshape(-1, 1) if your data has a single feature or X.reshape(1, -1) if it contains a single sample.
DeprecationWarning)
Loaded Data
Working on Data
Sample Vector [7, 13, 13, 17, 17, 3, 15, 3, 7, 6, 16, 11, 19, -0.8230769230769232, 50.43076923076922, 0.15384615384615385]
Balancing Data
Balanced Data
Finished working with Data
Training Data Points: 9235
Test Data Points: 2309
Starting Training
Done training
Test Results
Sensitivity: 0.02608695652173913
Specificity : 0.9990884229717412
Accuracy: 0.9506279774794283
ROC 0.512587689747
TP 3 FP 2 TN 2192 FN 112
None
Cross: Validation: [ 0.94891775 0.95062798 0.9532265 0.95060659 0.95017331]
Number of data points in benchmark 0
Benchmark not relevant
y near_miss Data/Benchmarks/phos_CDK1.csv
Loading Data
/Users/mark/anaconda3/lib/python3.6/site-packages/sklearn/utils/validation.py:395: DeprecationWarning: Passing 1d arrays as data is deprecated in 0.17 and will raise ValueError in 0.19. Reshape your data either using X.reshape(-1, 1) if your data has a single feature or X.reshape(1, -1) if it contains a single sample.
DeprecationWarning)
Loaded Data
Working on Data
Sample Vector [8, 12, 15, 18, 5, 13, 15, 19, 12, 10, 5, 6, 10, 0.22307692307692312, 4.284615384615385, 0.38461538461538464]
Balancing Data
Balanced Data
Finished working with Data
Training Data Points: 945
Test Data Points: 237
Starting Training
Done training
Test Results
Sensitivity: 0.5677966101694916
Specificity : 0.7142857142857143
Accuracy: 0.6413502109704642
ROC 0.641041162228
TP 67 FP 34 TN 85 FN 51
None
Cross: Validation: [ 0.6512605 0.63983051 0.63559322 0.58050847 0.70762712]
Number of data points in benchmark 0
Benchmark not relevant
y pass Data/Benchmarks/phos_CK2.csv
Loading Data
/Users/mark/anaconda3/lib/python3.6/site-packages/sklearn/utils/validation.py:395: DeprecationWarning: Passing 1d arrays as data is deprecated in 0.17 and will raise ValueError in 0.19. Reshape your data either using X.reshape(-1, 1) if your data has a single feature or X.reshape(1, -1) if it contains a single sample.
DeprecationWarning)
Loaded Data
Working on Data
Sample Vector [11, 11, 11, 7, 11, 17, 15, 2, 11, 3, 11, 9, 2, -1.1846153846153848, 82.67692307692309, 0.07692307692307693]
Finished working with Data
Training Data Points: 10099
Test Data Points: 2525
Starting Training
Done training
Test Results
Failed
TP 0 FP 7 TN 2402 FN 116
None
Cross: Validation: [ 0.95328583 0.95326733 0.95445545 0.95562599 0.95602219]
Number of data points in benchmark 398
Benchmark Results
Failed
TP 0 FP 1 TN 396 FN 1
None
x pass Data/Benchmarks/phos_CK2.csv
Loading Data
Loaded Data
Working on Data
Sample Vector [9, 20, 14, 15, 7, 13, 15, 9, 19, 9, 12, 11, 5, -0.676923076923077, 41.34615384615385, 0.23076923076923078]
Finished working with Data
Training Data Points: 10099
Test Data Points: 2525
Starting Training
Done training
Test Results
Sensitivity: 0.04580152671755725
Specificity : 0.9987468671679198
Accuracy: 0.9493069306930693
ROC 0.522274196943
TP 6 FP 3 TN 2391 FN 125
None
Cross: Validation: [ 0.95407759 0.95524752 0.95524752 0.95364501 0.9540412 ]
Number of data points in benchmark 398
Benchmark Results
Failed
TP 0 FP 0 TN 397 FN 1
None
y ADASYN Data/Benchmarks/phos_CK2.csv
Loading Data
Loaded Data
Working on Data
Sample Vector [20, 7, 9, 1, 18, 11, 15, 7, 20, 1, 8, 15, 1, -1.9307692307692306, -18.376923076923077, 0.15384615384615385]
Balancing Data
Balanced Data
Finished working with Data
Training Data Points: 19363
Test Data Points: 4841
Starting Training
Done training
Test Results
Sensitivity: 0.9406430338004946
Specificity : 0.9962732919254659
Accuracy: 0.9683949597190663
ROC 0.968458162863
TP 2282 FP 9 TN 2406 FN 144
None
Cross: Validation: [ 0.86699711 0.9787234 0.96984094 0.96838843 0.97561983]
Number of data points in benchmark 398
Benchmark Results
Failed
TP 0 FP 1 TN 396 FN 1
None
x ADASYN Data/Benchmarks/phos_CK2.csv
Loading Data
Loaded Data
Working on Data
Sample Vector [19, 17, 1, 17, 10, 16, 15, 5, 11, 1, 10, 3, 17, -1.453846153846154, 94.13076923076925, 0.15384615384615385]
Balancing Data
Balanced Data
Finished working with Data
Training Data Points: 19352
Test Data Points: 4838
Starting Training
Done training
Test Results
Sensitivity: 0.947324414715719
Specificity : 0.9975470155355682
Accuracy: 0.9727159983464242
ROC 0.972435715126
TP 2266 FP 6 TN 2440 FN 126
None
Cross: Validation: [ 0.86484811 0.98181442 0.979537 0.98097995 0.97415754]
Number of data points in benchmark 398
Benchmark Results
Failed
TP 0 FP 2 TN 395 FN 1
None
y SMOTEENN Data/Benchmarks/phos_CK2.csv
Loading Data
Loaded Data
Working on Data
Sample Vector [1, 20, 8, 20, 12, 18, 15, 12, 11, 10, 17, 9, 1, -0.9615384615384616, 41.34615384615385, 0.07692307692307693]
Balancing Data
Balanced Data
Finished working with Data
Training Data Points: 19231
Test Data Points: 4808
Starting Training
Done training
Test Results
Sensitivity: 0.942134590655808
Specificity : 0.9931313131313131
Accuracy: 0.9683860232945092
ROC 0.967632951894
TP 2198 FP 17 TN 2458 FN 135
None
Cross: Validation: [ 0.8889582 0.98772879 0.98731281 0.98398169 0.98689411]
Number of data points in benchmark 398
Benchmark Results
Failed
TP 0 FP 3 TN 394 FN 1
None
x SMOTEENN Data/Benchmarks/phos_CK2.csv
Loading Data
Loaded Data
Working on Data
Sample Vector [9, 2, 9, 2, 4, 18, 15, 9, 1, 10, 11, 13, 7, -0.9230769230769232, 30.7, 0.07692307692307693]
Balancing Data
Balanced Data
Finished working with Data
Training Data Points: 19222
Test Data Points: 4806
Starting Training
Done training
Test Results
Sensitivity: 0.9368061485909479
Specificity : 0.9886363636363636
Accuracy: 0.9633791094465252
ROC 0.962721256114
TP 2194 FP 28 TN 2436 FN 148
None
Cross: Validation: [ 0.89159384 0.98377029 0.98668331 0.98168574 0.98709677]
Number of data points in benchmark 398
Benchmark Results
Failed
TP 0 FP 6 TN 391 FN 1
None
y random_under_sample Data/Benchmarks/phos_CK2.csv
Loading Data
Loaded Data
Working on Data
Sample Vector [10, 9, 13, 9, 3, 9, 15, 20, 10, 11, 10, 11, 20, -0.8076923076923077, 148.54615384615386, 0.07692307692307693]
Balancing Data
Balanced Data
Finished working with Data
Training Data Points: 945
Test Data Points: 237
Starting Training
Done training
Test Results
Sensitivity: 0.4322033898305085
Specificity : 0.6554621848739496
Accuracy: 0.5443037974683544
ROC 0.543832787352
TP 51 FP 41 TN 78 FN 67
None
Cross: Validation: [ 0.59663866 0.58474576 0.59745763 0.56355932 0.61440678]
Number of data points in benchmark 398
Benchmark Results
Sensitivity: 1.0
Specificity : 0.6649874055415617
Accuracy: 0.6658291457286433
ROC 0.832493702771
TP 1 FP 133 TN 264 FN 0
None
x random_under_sample Data/Benchmarks/phos_CK2.csv
Loading Data
Loaded Data
Working on Data
Sample Vector [8, 10, 7, 17, 2, 7, 15, 5, 13, 18, 10, 7, 1, -1.3384615384615381, 61.57692307692308, 0.15384615384615385]
Balancing Data
Balanced Data
Finished working with Data
Training Data Points: 945
Test Data Points: 237
Starting Training
Done training
Test Results
Sensitivity: 0.4915254237288136
Specificity : 0.6974789915966386
Accuracy: 0.5949367088607594
ROC 0.594502207663
TP 58 FP 36 TN 83 FN 60
None
Cross: Validation: [ 0.57563025 0.56779661 0.56355932 0.53389831 0.63135593]
Number of data points in benchmark 398
Benchmark Results
Sensitivity: 1.0
Specificity : 0.7204030226700252
Accuracy: 0.7211055276381909
ROC 0.860201511335
TP 1 FP 111 TN 286 FN 0
None
y ncl Data/Benchmarks/phos_CK2.csv
Loading Data
Loaded Data
Working on Data
Sample Vector [15, 1, 17, 17, 16, 2, 15, 12, 10, 5, 7, 11, 14, -0.7846153846153846, 38.41538461538462, 0.23076923076923078]
Balancing Data
Balanced Data
Finished working with Data
Training Data Points: 9227
Test Data Points: 2307
Starting Training
Done training
Test Results
Sensitivity: 0.0782608695652174
Specificity : 0.9995437956204379
Accuracy: 0.9536194191590811
ROC 0.538902332593
TP 9 FP 1 TN 2191 FN 106
None
Cross: Validation: [ 0.95103986 0.94668401 0.95231903 0.95056375 0.9509974 ]
Number of data points in benchmark 398
Benchmark Results
Failed
TP 0 FP 0 TN 397 FN 1
None
x ncl Data/Benchmarks/phos_CK2.csv
Loading Data
Loaded Data
Working on Data
Sample Vector [12, 15, 10, 11, 3, 16, 15, 10, 2, 17, 11, 2, 10, -0.3307692307692307, 100.66153846153848, 0.15384615384615385]
Balancing Data
Balanced Data
Finished working with Data
Training Data Points: 9227
Test Data Points: 2307
Starting Training
Done training
Test Results
Sensitivity: 0.06956521739130435
Specificity : 0.9981751824817519
Accuracy: 0.951885565669701
ROC 0.533870199937
TP 8 FP 4 TN 2188 FN 107
None
Cross: Validation: [ 0.94844021 0.9514521 0.95101864 0.9518647 0.9501301 ]
Number of data points in benchmark 398
Benchmark Results
Failed
TP 0 FP 2 TN 395 FN 1
None
y near_miss Data/Benchmarks/phos_CK2.csv
Loading Data
Loaded Data
Working on Data
Sample Vector [4, 7, 3, 13, 13, 7, 15, 1, 4, 20, 12, 12, 8, 0.8692307692307693, -12.576923076923077, 0.07692307692307693]
Balancing Data
Balanced Data
Finished working with Data
Training Data Points: 945
Test Data Points: 237
Starting Training
Done training
Test Results
Sensitivity: 0.5
Specificity : 0.773109243697479
Accuracy: 0.6371308016877637
ROC 0.636554621849
TP 59 FP 27 TN 92 FN 59
None
Cross: Validation: [ 0.73109244 0.71610169 0.69067797 0.5720339 0.6440678 ]
Number of data points in benchmark 398
Benchmark Results
Failed
TP 0 FP 227 TN 170 FN 1
None
x near_miss Data/Benchmarks/phos_CK2.csv
Loading Data
Loaded Data
Working on Data
Sample Vector [5, 17, 3, 12, 9, 13, 15, 1, 13, 11, 9, 18, 13, 0.46153846153846156, 55.93846153846154, 0.15384615384615385]
Balancing Data
Balanced Data
Finished working with Data
Training Data Points: 945
Test Data Points: 237
Starting Training
Done training
Test Results
Sensitivity: 0.5932203389830508
Specificity : 0.7142857142857143
Accuracy: 0.6540084388185654
ROC 0.653753026634
TP 70 FP 34 TN 85 FN 48
None
Cross: Validation: [ 0.6512605 0.69491525 0.63135593 0.56355932 0.65677966]
Number of data points in benchmark 398
Benchmark Results
Sensitivity: 1.0
Specificity : 0.4534005037783375
Accuracy: 0.4547738693467337
ROC 0.726700251889
TP 1 FP 217 TN 180 FN 0
None
y pass Data/Benchmarks/phos_MAPK1.csv
Loading Data
Loaded Data
Working on Data
Sample Vector [2, 4, 11, 12, 13, 18, 15, 19, 3, 3, 14, 2, 12, 1.4307692307692308, 76.06153846153846, 0.07692307692307693]
Finished working with Data
Training Data Points: 10099
Test Data Points: 2525
Starting Training
Done training
Test Results
Sensitivity: 0.07352941176470588
Specificity : 0.9991628296358309
Accuracy: 0.9493069306930693
ROC 0.5363461207
TP 10 FP 2 TN 2387 FN 126
None
Cross: Validation: [ 0.95605701 0.95366337 0.95524752 0.95364501 0.95562599]
Number of data points in benchmark 0
Benchmark not relevant
y ADASYN Data/Benchmarks/phos_MAPK1.csv
Loading Data
/Users/mark/anaconda3/lib/python3.6/site-packages/sklearn/utils/validation.py:395: DeprecationWarning: Passing 1d arrays as data is deprecated in 0.17 and will raise ValueError in 0.19. Reshape your data either using X.reshape(-1, 1) if your data has a single feature or X.reshape(1, -1) if it contains a single sample.
DeprecationWarning)
Loaded Data
Working on Data
Sample Vector [11, 1, 20, 2, 8, 9, 15, 1, 1, 13, 13, 17, 16, -0.6692307692307694, 28.761538461538464, 0.07692307692307693]
Balancing Data
Balanced Data
Finished working with Data
Training Data Points: 19356
Test Data Points: 4840
Starting Training
Done training
Test Results
Sensitivity: 0.9442851290454731
Specificity : 0.9941642350979575
Accuracy: 0.96900826446281
ROC 0.969224682072
TP 2305 FP 14 TN 2385 FN 136
None
Cross: Validation: [ 0.87376033 0.97789256 0.9822314 0.97560976 0.97788342]
Number of data points in benchmark 0
Benchmark not relevant
y SMOTEENN Data/Benchmarks/phos_MAPK1.csv
Loading Data
/Users/mark/anaconda3/lib/python3.6/site-packages/sklearn/utils/validation.py:395: DeprecationWarning: Passing 1d arrays as data is deprecated in 0.17 and will raise ValueError in 0.19. Reshape your data either using X.reshape(-1, 1) if your data has a single feature or X.reshape(1, -1) if it contains a single sample.
DeprecationWarning)
Loaded Data
Working on Data
Sample Vector [1, 14, 4, 1, 9, 20, 15, 5, 19, 18, 7, 11, 13, -0.5461538461538461, 50.415384615384625, 0.15384615384615385]
Balancing Data
Balanced Data
Finished working with Data
Training Data Points: 19222
Test Data Points: 4806
Starting Training
Done training
Test Results
Sensitivity: 0.9530315969257045
Specificity : 0.9886363636363636
Accuracy: 0.9712858926342073
ROC 0.970833980281
TP 2232 FP 28 TN 2436 FN 110
None
Cross: Validation: [ 0.89180191 0.98709946 0.9852268 0.98376691 0.98210198]
Number of data points in benchmark 0
Benchmark not relevant
y random_under_sample Data/Benchmarks/phos_MAPK1.csv
Loading Data
/Users/mark/anaconda3/lib/python3.6/site-packages/sklearn/utils/validation.py:395: DeprecationWarning: Passing 1d arrays as data is deprecated in 0.17 and will raise ValueError in 0.19. Reshape your data either using X.reshape(-1, 1) if your data has a single feature or X.reshape(1, -1) if it contains a single sample.
DeprecationWarning)
Loaded Data
Working on Data
Sample Vector [10, 12, 7, 12, 15, 10, 15, 15, 18, 3, 9, 9, 10, -0.6538461538461539, 44.06923076923078, 0.23076923076923078]
Balancing Data
Balanced Data
Finished working with Data
Training Data Points: 945
Test Data Points: 237
Starting Training
Done training
Test Results
Sensitivity: 0.4915254237288136
Specificity : 0.6638655462184874
Accuracy: 0.5780590717299579
ROC 0.577695484974
TP 58 FP 40 TN 79 FN 60
None
Cross: Validation: [ 0.6092437 0.52966102 0.59322034 0.63559322 0.61440678]
Number of data points in benchmark 0
Benchmark not relevant
y ncl Data/Benchmarks/phos_MAPK1.csv
Loading Data
/Users/mark/anaconda3/lib/python3.6/site-packages/sklearn/utils/validation.py:395: DeprecationWarning: Passing 1d arrays as data is deprecated in 0.17 and will raise ValueError in 0.19. Reshape your data either using X.reshape(-1, 1) if your data has a single feature or X.reshape(1, -1) if it contains a single sample.
DeprecationWarning)
Loaded Data
Working on Data
Sample Vector [20, 5, 5, 11, 2, 12, 15, 18, 12, 10, 12, 10, 9, 0.5999999999999999, 57.36923076923077, 0.23076923076923078]
Balancing Data
Balanced Data
Finished working with Data
Training Data Points: 9225
Test Data Points: 2307
Starting Training
Done training
Test Results
Sensitivity: 0.043478260869565216
Specificity : 0.9968065693430657
Accuracy: 0.9492847854356307
ROC 0.520142415106
TP 5 FP 7 TN 2185 FN 110
None
Cross: Validation: [ 0.95320624 0.95143105 0.94622723 0.95143105 0.9518647 ]
Number of data points in benchmark 0
Benchmark not relevant
y near_miss Data/Benchmarks/phos_MAPK1.csv
Loading Data
/Users/mark/anaconda3/lib/python3.6/site-packages/sklearn/utils/validation.py:395: DeprecationWarning: Passing 1d arrays as data is deprecated in 0.17 and will raise ValueError in 0.19. Reshape your data either using X.reshape(-1, 1) if your data has a single feature or X.reshape(1, -1) if it contains a single sample.
DeprecationWarning)
Loaded Data
Working on Data
Sample Vector [10, 7, 3, 10, 5, 14, 15, 3, 14, 5, 12, 10, 13, 1.4846153846153844, 2.7, 0.23076923076923078]
Balancing Data
Balanced Data
Finished working with Data
Training Data Points: 945
Test Data Points: 237
Starting Training
Done training
Test Results
Sensitivity: 0.6101694915254238
Specificity : 0.7058823529411765
Accuracy: 0.6582278481012658
ROC 0.658025922233
TP 72 FP 35 TN 84 FN 46
None
Cross: Validation: [ 0.66386555 0.74152542 0.65254237 0.60169492 0.58474576]
Number of data points in benchmark 0
Benchmark not relevant
y pass Data/Benchmarks/phos_PKA.csv
Loading Data
/Users/mark/anaconda3/lib/python3.6/site-packages/sklearn/utils/validation.py:395: DeprecationWarning: Passing 1d arrays as data is deprecated in 0.17 and will raise ValueError in 0.19. Reshape your data either using X.reshape(-1, 1) if your data has a single feature or X.reshape(1, -1) if it contains a single sample.
DeprecationWarning)
Loaded Data
Working on Data
Sample Vector [3, 8, 3, 19, 18, 8, 15, 2, 12, 3, 9, 18, 8, -0.6461538461538462, 35.12307692307693, 0.07692307692307693]
Finished working with Data
Training Data Points: 10099
Test Data Points: 2525
Starting Training
Done training
Test Results
Sensitivity: 0.035398230088495575
Specificity : 0.9995854063018242
Accuracy: 0.9564356435643564
ROC 0.517491818195
TP 4 FP 1 TN 2411 FN 109
None
Cross: Validation: [ 0.95407759 0.95485149 0.95445545 0.95364501 0.95522979]
Number of data points in benchmark 0
Benchmark not relevant
y ADASYN Data/Benchmarks/phos_PKA.csv
Loading Data
/Users/mark/anaconda3/lib/python3.6/site-packages/sklearn/utils/validation.py:395: DeprecationWarning: Passing 1d arrays as data is deprecated in 0.17 and will raise ValueError in 0.19. Reshape your data either using X.reshape(-1, 1) if your data has a single feature or X.reshape(1, -1) if it contains a single sample.
DeprecationWarning)
Loaded Data
Working on Data
Sample Vector [10, 11, 20, 10, 11, 10, 15, 10, 11, 20, 10, 11, 18, -1.276923076923077, 159.24615384615385, 0.07692307692307693]
Balancing Data
Balanced Data
Finished working with Data
Training Data Points: 19348
Test Data Points: 4838
Starting Training
Done training
Test Results
Sensitivity: 0.9430590191188695
Specificity : 0.9958881578947368
Accuracy: 0.9696155436130632
ROC 0.969473588507
TP 2269 FP 10 TN 2422 FN 137
None
Cross: Validation: [ 0.86440678 0.97747003 0.98181067 0.97663358 0.97105045]
Number of data points in benchmark 0
Benchmark not relevant
y SMOTEENN Data/Benchmarks/phos_PKA.csv
Loading Data
/Users/mark/anaconda3/lib/python3.6/site-packages/sklearn/utils/validation.py:395: DeprecationWarning: Passing 1d arrays as data is deprecated in 0.17 and will raise ValueError in 0.19. Reshape your data either using X.reshape(-1, 1) if your data has a single feature or X.reshape(1, -1) if it contains a single sample.
DeprecationWarning)
Loaded Data
Working on Data
Sample Vector [17, 17, 17, 5, 6, 19, 15, 1, 17, 13, 2, 3, 12, -0.5384615384615381, 90.82307692307694, 0.23076923076923078]
Balancing Data
Balanced Data
Finished working with Data
Training Data Points: 15879
Test Data Points: 3970
Starting Training
Done training
Test Results
Sensitivity: 0.9572719802793755
Specificity : 0.984375
Accuracy: 0.9677581863979848
ROC 0.97082349014
TP 2330 FP 24 TN 1512 FN 104
None
Cross: Validation: [ 0.87660539 0.97304786 0.98211587 0.9808516 0.98034769]
Number of data points in benchmark 0
Benchmark not relevant
y random_under_sample Data/Benchmarks/phos_PKA.csv
Loading Data
/Users/mark/anaconda3/lib/python3.6/site-packages/sklearn/utils/validation.py:395: DeprecationWarning: Passing 1d arrays as data is deprecated in 0.17 and will raise ValueError in 0.19. Reshape your data either using X.reshape(-1, 1) if your data has a single feature or X.reshape(1, -1) if it contains a single sample.
DeprecationWarning)
Loaded Data
Working on Data
Sample Vector [17, 9, 20, 8, 20, 4, 15, 7, 2, 10, 8, 9, 10, -1.7692307692307692, 66.09230769230771, 0.07692307692307693]
Balancing Data
Balanced Data
Finished working with Data
Training Data Points: 945
Test Data Points: 237
Starting Training
Done training
Test Results
Sensitivity: 0.4830508474576271
Specificity : 0.6554621848739496
Accuracy: 0.569620253164557
ROC 0.569256516166
TP 57 FP 41 TN 78 FN 61
None
Cross: Validation: [ 0.57983193 0.61016949 0.6059322 0.62288136 0.55084746]
Number of data points in benchmark 0
Benchmark not relevant
y ncl Data/Benchmarks/phos_PKA.csv
Loading Data
/Users/mark/anaconda3/lib/python3.6/site-packages/sklearn/utils/validation.py:395: DeprecationWarning: Passing 1d arrays as data is deprecated in 0.17 and will raise ValueError in 0.19. Reshape your data either using X.reshape(-1, 1) if your data has a single feature or X.reshape(1, -1) if it contains a single sample.
DeprecationWarning)
Loaded Data
Working on Data
Sample Vector [9, 19, 19, 18, 9, 5, 18, 3, 12, 15, 9, 18, 3, -1.1307692307692307, 12.44615384615385, 0.15384615384615385]
Balancing Data
Balanced Data
Finished working with Data
Training Data Points: 9233
Test Data Points: 2309
Starting Training
Done training
Test Results
Sensitivity: 0.0782608695652174
Specificity : 0.9995442114858706
Accuracy: 0.9536595928973581
ROC 0.538902540526
TP 9 FP 1 TN 2193 FN 106
None
Cross: Validation: [ 0.94935065 0.95060659 0.95147314 0.94930676 0.95017331]
Number of data points in benchmark 0
Benchmark not relevant
y near_miss Data/Benchmarks/phos_PKA.csv
Loading Data
/Users/mark/anaconda3/lib/python3.6/site-packages/sklearn/utils/validation.py:395: DeprecationWarning: Passing 1d arrays as data is deprecated in 0.17 and will raise ValueError in 0.19. Reshape your data either using X.reshape(-1, 1) if your data has a single feature or X.reshape(1, -1) if it contains a single sample.
DeprecationWarning)
Loaded Data
Working on Data
Sample Vector [18, 9, 4, 20, 18, 14, 15, 20, 18, 8, 18, 20, 1, -1.5692307692307692, -43.71538461538462, 0.07692307692307693]
Balancing Data
Balanced Data
Finished working with Data
Training Data Points: 945
Test Data Points: 237
Starting Training
Done training
Test Results
Sensitivity: 0.5847457627118644
Specificity : 0.7394957983193278
Accuracy: 0.6624472573839663
ROC 0.662120780516
TP 69 FP 31 TN 88 FN 49
None
Cross: Validation: [ 0.62184874 0.69915254 0.63983051 0.62711864 0.62711864]
Number of data points in benchmark 0
Benchmark not relevant
y pass Data/Benchmarks/phos_PKC.csv
Loading Data
/Users/mark/anaconda3/lib/python3.6/site-packages/sklearn/utils/validation.py:395: DeprecationWarning: Passing 1d arrays as data is deprecated in 0.17 and will raise ValueError in 0.19. Reshape your data either using X.reshape(-1, 1) if your data has a single feature or X.reshape(1, -1) if it contains a single sample.
DeprecationWarning)
Loaded Data
Working on Data
Sample Vector [1, 14, 19, 9, 19, 3, 15, 9, 10, 4, 2, 2, 5, -0.14615384615384625, 57.36923076923078, 0.15384615384615385]
Finished working with Data
Training Data Points: 10099
Test Data Points: 2525
Starting Training
Done training
Test Results
Sensitivity: 0.06481481481481481
Specificity : 0.998345055854365
Accuracy: 0.9584158415841584
ROC 0.531579935335
TP 7 FP 4 TN 2413 FN 101
None
Cross: Validation: [ 0.95605701 0.95445545 0.95524752 0.95522979 0.9544374 ]
Number of data points in benchmark 569
Benchmark Results
Sensitivity: 1.0
Specificity : 0.9964788732394366
Accuracy: 0.9964850615114236
ROC 0.99823943662
TP 1 FP 2 TN 566 FN 0
None
x pass Data/Benchmarks/phos_PKC.csv
Loading Data
Loaded Data
Working on Data
Sample Vector [3, 18, 12, 9, 11, 19, 15, 3, 9, 12, 3, 9, 7, -0.34615384615384615, 18.24615384615385, 0.07692307692307693]
Finished working with Data
Training Data Points: 10099
Test Data Points: 2525
Starting Training
Done training
Test Results
Sensitivity: 0.0743801652892562
Specificity : 0.9991680532445923
Accuracy: 0.9548514851485148
ROC 0.536774109267
TP 9 FP 2 TN 2402 FN 112
None
Cross: Validation: [ 0.95566112 0.95287129 0.95485149 0.95641838 0.9540412 ]
Number of data points in benchmark 569
Benchmark Results
Sensitivity: 1.0
Specificity : 0.9947183098591549
Accuracy: 0.9947275922671354
ROC 0.99735915493
TP 1 FP 3 TN 565 FN 0
None
y ADASYN Data/Benchmarks/phos_PKC.csv
Loading Data
Loaded Data
Working on Data
Sample Vector [11, 5, 14, 1, 7, 14, 15, 16, 11, 9, 3, 8, 7, -0.8692307692307693, 63.23846153846155, 0.15384615384615385]
Balancing Data
Balanced Data
Finished working with Data
Training Data Points: 19356
Test Data Points: 4840
Starting Training
Done training
Test Results
Sensitivity: 0.9516528925619835
Specificity : 0.9975206611570248
Accuracy: 0.9745867768595041
ROC 0.97458677686
TP 2303 FP 6 TN 2414 FN 117
None
Cross: Validation: [ 0.86012397 0.97830579 0.97603306 0.97540306 0.98491112]
Number of data points in benchmark 569
Benchmark Results
Sensitivity: 1.0
Specificity : 0.9964788732394366
Accuracy: 0.9964850615114236
ROC 0.99823943662
TP 1 FP 2 TN 566 FN 0
None
x ADASYN Data/Benchmarks/phos_PKC.csv
Loading Data
Loaded Data
Working on Data
Sample Vector [10, 9, 9, 19, 18, 11, 15, 20, 11, 20, 15, 8, 5, -1.7461538461538464, 49.3923076923077, 0.23076923076923078]
Balancing Data
Balanced Data
Finished working with Data
Training Data Points: 19356
Test Data Points: 4840
Starting Training
Done training
Test Results
Sensitivity: 0.9516528925619835
Specificity : 0.9975206611570248
Accuracy: 0.9745867768595041
ROC 0.97458677686
TP 2303 FP 6 TN 2414 FN 117
None
Cross: Validation: [ 0.86590909 0.97458678 0.96756198 0.98243076 0.97891691]
Number of data points in benchmark 569
Benchmark Results
Sensitivity: 1.0
Specificity : 0.9964788732394366
Accuracy: 0.9964850615114236
ROC 0.99823943662
TP 1 FP 2 TN 566 FN 0
None
y SMOTEENN Data/Benchmarks/phos_PKC.csv
Loading Data
Loaded Data
Working on Data
Sample Vector [9, 19, 9, 9, 1, 11, 15, 15, 10, 2, 3, 10, 1, -1.153846153846154, 58.61538461538462, 0.15384615384615385]
Balancing Data
Balanced Data
Finished working with Data
Training Data Points: 19225
Test Data Points: 4807
Starting Training
Done training
Test Results
Sensitivity: 0.9340380549682875
Specificity : 0.9877149877149877
Accuracy: 0.9613064281256501
ROC 0.960876521342
TP 2209 FP 30 TN 2412 FN 156
None
Cross: Validation: [ 0.89494487 0.9833576 0.97982109 0.98813983 0.98376691]
Number of data points in benchmark 569
Benchmark Results
Sensitivity: 1.0
Specificity : 0.9929577464788732
Accuracy: 0.9929701230228472
ROC 0.996478873239
TP 1 FP 4 TN 564 FN 0
None
x SMOTEENN Data/Benchmarks/phos_PKC.csv
Loading Data
Loaded Data
Working on Data
Sample Vector [12, 1, 3, 15, 2, 12, 15, 3, 3, 19, 6, 3, 10, 1.323076923076923, 24.607692307692307, 0.23076923076923078]
Balancing Data
Balanced Data
Finished working with Data
Training Data Points: 19239
Test Data Points: 4810
Starting Training
Done training
Test Results
Sensitivity: 0.9411519198664441
Specificity : 0.9888152444076223
Accuracy: 0.9650727650727651
ROC 0.964983582137
TP 2255 FP 27 TN 2387 FN 141
None
Cross: Validation: [ 0.89503222 0.98295218 0.98295218 0.98877105 0.98585985]
Number of data points in benchmark 569
Benchmark Results
Sensitivity: 1.0
Specificity : 0.9911971830985915
Accuracy: 0.9912126537785588
ROC 0.995598591549
TP 1 FP 5 TN 563 FN 0
None
y random_under_sample Data/Benchmarks/phos_PKC.csv
Loading Data
Loaded Data
Working on Data
Sample Vector [7, 7, 19, 2, 9, 7, 15, 2, 7, 5, 5, 9, 19, -1.6692307692307689, 42.261538461538464, 0.23076923076923078]
Balancing Data
Balanced Data
Finished working with Data
Training Data Points: 945
Test Data Points: 237
Starting Training
Done training
Test Results
Sensitivity: 0.6271186440677966
Specificity : 0.6722689075630253
Accuracy: 0.6497890295358649
ROC 0.649693775815
TP 74 FP 39 TN 80 FN 44
None
Cross: Validation: [ 0.60504202 0.61440678 0.55932203 0.55508475 0.57627119]
Number of data points in benchmark 569
Benchmark Results
Sensitivity: 1.0
Specificity : 0.6602112676056338
Accuracy: 0.6608084358523726
ROC 0.830105633803
TP 1 FP 193 TN 375 FN 0
None
x random_under_sample Data/Benchmarks/phos_PKC.csv
Loading Data
Loaded Data
Working on Data
Sample Vector [9, 5, 3, 2, 17, 12, 15, 9, 7, 3, 13, 20, 1, 0.23846153846153847, -15.430769230769233, 0.15384615384615385]
Balancing Data
Balanced Data
Finished working with Data
Training Data Points: 945
Test Data Points: 237
Starting Training
Done training
Test Results
Sensitivity: 0.4915254237288136
Specificity : 0.6638655462184874
Accuracy: 0.5780590717299579
ROC 0.577695484974
TP 58 FP 40 TN 79 FN 60
None
Cross: Validation: [ 0.56722689 0.60169492 0.56355932 0.58474576 0.62711864]
Number of data points in benchmark 569
Benchmark Results
Sensitivity: 1.0
Specificity : 0.6690140845070423
Accuracy: 0.6695957820738138
ROC 0.834507042254
TP 1 FP 188 TN 380 FN 0
None
y ncl Data/Benchmarks/phos_PKC.csv
Loading Data
Loaded Data
Working on Data
Sample Vector [20, 6, 7, 19, 2, 11, 15, 15, 13, 18, 3, 12, 9, -0.453846153846154, 21.97692307692308, 0.23076923076923078]
Balancing Data
Balanced Data
Finished working with Data
Training Data Points: 9227
Test Data Points: 2307
Starting Training
Done training
Test Results
Sensitivity: 0.11304347826086956
Specificity : 0.9977189781021898
Accuracy: 0.9536194191590811
ROC 0.555381228182
TP 13 FP 5 TN 2187 FN 102
None
Cross: Validation: [ 0.95103986 0.95101864 0.95015171 0.9501301 0.95056375]
Number of data points in benchmark 569
Benchmark Results
Sensitivity: 1.0
Specificity : 0.9964788732394366
Accuracy: 0.9964850615114236
ROC 0.99823943662
TP 1 FP 2 TN 566 FN 0
None
x ncl Data/Benchmarks/phos_PKC.csv
Loading Data
Loaded Data
Working on Data
Sample Vector [18, 18, 2, 10, 20, 19, 15, 19, 3, 10, 19, 7, 10, -1.5538461538461539, 20.915384615384614, 0.07692307692307693]
Balancing Data
Balanced Data
Finished working with Data
Training Data Points: 9228
Test Data Points: 2308
Starting Training
Done training
Test Results
Sensitivity: 0.06086956521739131
Specificity : 0.9990880072959416
Accuracy: 0.9523396880415944
ROC 0.529978786257
TP 7 FP 2 TN 2191 FN 108
None
Cross: Validation: [ 0.94887348 0.95361942 0.9514521 0.94971825 0.95275249]
Number of data points in benchmark 569
Benchmark Results
Sensitivity: 1.0
Specificity : 0.9947183098591549
Accuracy: 0.9947275922671354
ROC 0.99735915493
TP 1 FP 3 TN 565 FN 0
None
y near_miss Data/Benchmarks/phos_PKC.csv
Loading Data
Loaded Data
Working on Data
Sample Vector [19, 11, 18, 10, 11, 3, 15, 10, 12, 7, 10, 5, 9, -0.8076923076923077, 40.815384615384616, 0.15384615384615385]
Balancing Data
Balanced Data
Finished working with Data
Training Data Points: 945
Test Data Points: 237
Starting Training
Done training
Test Results
Sensitivity: 0.5338983050847458
Specificity : 0.7478991596638656
Accuracy: 0.6413502109704642
ROC 0.640898732374
TP 63 FP 30 TN 89 FN 55
None
Cross: Validation: [ 0.67647059 0.6440678 0.61864407 0.65254237 0.63559322]
Number of data points in benchmark 569
Benchmark Results
Sensitivity: 1.0
Specificity : 0.44366197183098594
Accuracy: 0.4446397188049209
ROC 0.721830985915
TP 1 FP 316 TN 252 FN 0
None
x near_miss Data/Benchmarks/phos_PKC.csv
Loading Data
Loaded Data
Working on Data
Sample Vector [11, 11, 2, 11, 16, 10, 15, 13, 18, 9, 6, 3, 8, -0.8769230769230771, 76.68461538461538, 0.15384615384615385]
Balancing Data
Balanced Data
Finished working with Data
Training Data Points: 945
Test Data Points: 237
Starting Training
Done training
Test Results
Sensitivity: 0.6016949152542372
Specificity : 0.6974789915966386
Accuracy: 0.6497890295358649
ROC 0.649586953425
TP 71 FP 36 TN 83 FN 47
None
Cross: Validation: [ 0.65966387 0.63559322 0.6440678 0.56355932 0.64830508]
Number of data points in benchmark 569
Benchmark Results
Sensitivity: 1.0
Specificity : 0.4630281690140845
Accuracy: 0.46397188049209137
ROC 0.731514084507
TP 1 FP 305 TN 263 FN 0
None
T Phosphorylation
In [4]:
par = ["pass", "ADASYN", "SMOTEENN", "random_under_sample", "ncl", "near_miss"]
benchmarks = ["Data/Benchmarks/phos_CDK1.csv", "Data/Benchmarks/phos_CK2.csv", "Data/Benchmarks/phos_MAPK1.csv", "Data/Benchmarks/phos_PKA.csv", "Data/Benchmarks/phos_PKC.csv"]
for j in benchmarks:
for i in par:
print("y", i, " ", j)
y = Predictor()
y.load_data(file="Data/Training/clean_t_filtered.csv")
y.process_data(vector_function="sequence", amino_acid="T", imbalance_function=i, random_data=0)
y.supervised_training("bagging")
y.benchmark(j, "T")
del y
print("x", i, " ", j)
x = Predictor()
x.load_data(file="Data/Training/clean_t_filtered.csv")
x.process_data(vector_function="sequence", amino_acid="T", imbalance_function=i, random_data=1)
x.supervised_training("bagging")
x.benchmark(j, "T")
del x
y pass Data/Benchmarks/phos_CDK1.csv
Loading Data
Loaded Data
Working on Data
Sample Vector [19, 10, 10, 1, 1, 20, 20, 2, 3, 17, 11, 10, 2, -0.5230769230769232, 86.26923076923077, 0.0]
Finished working with Data
Training Data Points: 66323
Test Data Points: 16581
Starting Training
Done training
Test Results
Sensitivity: 0.09866056844168572
Specificity : 0.9607988165680473
Accuracy: 0.8016404318195525
ROC 0.529729692505
TP 302 FP 530 TN 12990 FN 2759
None
Cross: Validation: [ 0.80328067 0.8035822 0.8048854 0.80410133 0.80211098]
Number of data points in benchmark 1196
Benchmark Results
Sensitivity: 0.32558139534883723
Specificity : 0.9496964440589766
Accuracy: 0.927257525083612
ROC 0.637638919704
TP 14 FP 58 TN 1095 FN 29
None
x pass Data/Benchmarks/phos_CDK1.csv
Loading Data
Loaded Data
Working on Data
Sample Vector [6, 1, 10, 11, 7, 12, 20, 7, 19, 1, 12, 20, 12, -0.3230769230769231, 15.461538461538458, 0.07692307692307693]
Finished working with Data
Training Data Points: 66323
Test Data Points: 16581
Starting Training
Done training
Test Results
Sensitivity: 0.10491146007350484
Specificity : 0.9596702973211657
Accuracy: 0.8053796514082383
ROC 0.532290878697
TP 314 FP 548 TN 13040 FN 2679
None
Cross: Validation: [ 0.80490894 0.80593415 0.8035585 0.80120627 0.80603136]
Number of data points in benchmark 1196
Benchmark Results
Sensitivity: 0.4418604651162791
Specificity : 0.9488291413703382
Accuracy: 0.9306020066889632
ROC 0.695344803243
TP 19 FP 59 TN 1094 FN 24
None
y ADASYN Data/Benchmarks/phos_CDK1.csv
Loading Data
Loaded Data
Working on Data
Sample Vector [7, 17, 3, 7, 18, 20, 3, 20, 8, 20, 20, 9, 0, -1.4999999999999998, 31.950000000000003, 0.0]
Balancing Data
Balanced Data
Finished working with Data
Training Data Points: 109779
Test Data Points: 27445
Starting Training
Done training
Test Results
Sensitivity: 0.8000578913090672
Specificity : 0.9451783355350066
Accuracy: 0.8721078520677719
ROC 0.872618113422
TP 11056 FP 747 TN 12879 FN 2763
None
Cross: Validation: [ 0.52124171 0.91641769 0.94425011 0.94752951 0.94541612]
Number of data points in benchmark 1196
Benchmark Results
Sensitivity: 0.4418604651162791
Specificity : 0.9375542064180399
Accuracy: 0.919732441471572
ROC 0.689707335767
TP 19 FP 72 TN 1081 FN 24
None
x ADASYN Data/Benchmarks/phos_CDK1.csv
Loading Data
Loaded Data
Working on Data
Sample Vector [1, 19, 12, 10, 1, 15, 20, 3, 17, 8, 19, 13, 11, -0.5923076923076922, 38.400000000000006, 0.07692307692307693]
Balancing Data
Balanced Data
Finished working with Data
Training Data Points: 109804
Test Data Points: 27451
Starting Training
Done training
Test Results
Sensitivity: 0.7970712798794056
Specificity : 0.9469674556213018
Accuracy: 0.8708972350734036
ROC 0.87201936775
TP 11104 FP 717 TN 12803 FN 2827
None
Cross: Validation: [ 0.52065423 0.91672738 0.94681432 0.94495446 0.94422587]
Number of data points in benchmark 1196
Benchmark Results
Sensitivity: 0.5348837209302325
Specificity : 0.9436253252385083
Accuracy: 0.9289297658862876
ROC 0.739254523084
TP 23 FP 65 TN 1088 FN 20
None
y SMOTEENN Data/Benchmarks/phos_CDK1.csv
Loading Data
Loaded Data
Working on Data
Sample Vector [9, 9, 7, 20, 20, 20, 20, 3, 13, 9, 11, 13, 17, -0.8076923076923076, 82.92307692307692, 0.0]
Balancing Data
Balanced Data
Finished working with Data
Training Data Points: 102951
Test Data Points: 25738
Starting Training
Done training
Test Results
Sensitivity: 0.8035655871769709
Specificity : 0.9624722427831236
Accuracy: 0.8869764550470122
ROC 0.88301891498
TP 9826 FP 507 TN 13003 FN 2402
None
Cross: Validation: [ 0.56086095 0.94580209 0.95446245 0.95438474 0.95275285]
Number of data points in benchmark 1196
Benchmark Results
Sensitivity: 0.32558139534883723
Specificity : 0.9418907198612315
Accuracy: 0.919732441471572
ROC 0.633736057605
TP 14 FP 67 TN 1086 FN 29
None
x SMOTEENN Data/Benchmarks/phos_CDK1.csv
Loading Data
Loaded Data
Working on Data
Sample Vector [10, 15, 9, 4, 10, 20, 20, 20, 20, 2, 18, 20, 10, -0.8076923076923077, 30.7, 0.07692307692307693]
Balancing Data
Balanced Data
Finished working with Data
Training Data Points: 102974
Test Data Points: 25744
Starting Training
Done training
Test Results
Sensitivity: 0.8066180686378901
Specificity : 0.960029553010713
Accuracy: 0.8872747047855811
ROC 0.883323810824
TP 9848 FP 541 TN 12994 FN 2361
None
Cross: Validation: [ 0.56193436 0.94763828 0.953735 0.9545896 0.95591034]
Number of data points in benchmark 1196
Benchmark Results
Sensitivity: 0.4883720930232558
Specificity : 0.9418907198612315
Accuracy: 0.9255852842809364
ROC 0.715131406442
TP 21 FP 67 TN 1086 FN 22
None
y random_under_sample Data/Benchmarks/phos_CDK1.csv
Loading Data
Loaded Data
Working on Data
Sample Vector [15, 3, 8, 2, 2, 8, 20, 12, 11, 8, 17, 3, 11, -0.3692307692307693, 78.75384615384617, 0.07692307692307693]
Balancing Data
Balanced Data
Finished working with Data
Training Data Points: 24059
Test Data Points: 6015
Starting Training
Done training
Test Results
Sensitivity: 0.5536184210526316
Specificity : 0.6971428571428572
Accuracy: 0.6246051537822112
ROC 0.625380639098
TP 1683 FP 901 TN 2074 FN 1357
None
Cross: Validation: [ 0.62765957 0.6299867 0.62254739 0.62537413 0.61972065]
Number of data points in benchmark 1196
Benchmark Results
Sensitivity: 0.7674418604651163
Specificity : 0.6331309627059843
Accuracy: 0.6379598662207357
ROC 0.700286411586
TP 33 FP 423 TN 730 FN 10
None
x random_under_sample Data/Benchmarks/phos_CDK1.csv
Loading Data
Loaded Data
Working on Data
Sample Vector [10, 2, 9, 1, 17, 4, 20, 10, 1, 19, 7, 10, 8, -1.4692307692307693, 15.3, 0.0]
Balancing Data
Balanced Data
Finished working with Data
Training Data Points: 24059
Test Data Points: 6015
Starting Training
Done training
Test Results
Sensitivity: 0.5375
Specificity : 0.6974789915966386
Accuracy: 0.6166251039068994
ROC 0.617489495798
TP 1634 FP 900 TN 2075 FN 1406
None
Cross: Validation: [ 0.6221742 0.61685505 0.60808114 0.60974393 0.61822414]
Number of data points in benchmark 1196
Benchmark Results
Sensitivity: 0.9069767441860465
Specificity : 0.6079791847354726
Accuracy: 0.6187290969899666
ROC 0.757477964461
TP 39 FP 452 TN 701 FN 4
None
y ncl Data/Benchmarks/phos_CDK1.csv
Loading Data
Loaded Data
Working on Data
Sample Vector [5, 11, 11, 20, 20, 11, 20, 15, 11, 1, 4, 20, 11, -0.6000000000000001, 46.13846153846154, 0.15384615384615385]
Balancing Data
Balanced Data
Finished working with Data
Training Data Points: 47270
Test Data Points: 11818
Starting Training
Done training
Test Results
Sensitivity: 0.2366541992519551
Specificity : 0.9348879125830799
Accuracy: 0.7611270942629886
ROC 0.585771055918
TP 696 FP 578 TN 8299 FN 2245
None
Cross: Validation: [ 0.75446315 0.75664241 0.75721418 0.75687569 0.75882204]
Number of data points in benchmark 1196
Benchmark Results
Sensitivity: 0.5581395348837209
Specificity : 0.9045967042497832
Accuracy: 0.8921404682274248
ROC 0.731368119567
TP 24 FP 110 TN 1043 FN 19
None
x ncl Data/Benchmarks/phos_CDK1.csv
Loading Data
Loaded Data
Working on Data
Sample Vector [12, 14, 9, 13, 6, 3, 20, 10, 9, 19, 20, 1, 2, 0.21538461538461537, 38.54615384615385, 0.07692307692307693]
Balancing Data
Balanced Data
Finished working with Data
Training Data Points: 47234
Test Data Points: 11809
Starting Training
Done training
Test Results
Sensitivity: 0.23639455782312926
Specificity : 0.9290788138459803
Accuracy: 0.7566263019730713
ROC 0.582736685835
TP 695 FP 629 TN 8240 FN 2245
None
Cross: Validation: [ 0.75385267 0.76026759 0.75838415 0.75321816 0.75677507]
Number of data points in benchmark 1196
Benchmark Results
Sensitivity: 0.5813953488372093
Specificity : 0.8976582827406765
Accuracy: 0.8862876254180602
ROC 0.739526815789
TP 25 FP 118 TN 1035 FN 18
None
y near_miss Data/Benchmarks/phos_CDK1.csv
Loading Data
Loaded Data
Working on Data
Sample Vector [19, 15, 11, 15, 5, 9, 20, 10, 2, 7, 19, 2, 20, -1.1076923076923075, 18.723076923076924, 0.23076923076923078]
Balancing Data
Balanced Data
Finished working with Data
Training Data Points: 24059
Test Data Points: 6015
Starting Training
Done training
Test Results
Sensitivity: 0.5398026315789474
Specificity : 0.6810084033613445
Accuracy: 0.6096425602660016
ROC 0.61040551747
TP 1641 FP 949 TN 2026 FN 1399
None
Cross: Validation: [ 0.53989362 0.64860372 0.62770203 0.60957765 0.59860326]
Number of data points in benchmark 1196
Benchmark Results
Sensitivity: 0.6511627906976745
Specificity : 0.5056374674761491
Accuracy: 0.5108695652173914
ROC 0.578400129087
TP 28 FP 570 TN 583 FN 15
None
x near_miss Data/Benchmarks/phos_CDK1.csv
Loading Data
Loaded Data
Working on Data
Sample Vector [1, 11, 20, 9, 1, 5, 20, 9, 7, 18, 12, 11, 9, -1.2538461538461536, 67.04615384615384, 0.07692307692307693]
Balancing Data
Balanced Data
Finished working with Data
Training Data Points: 24059
Test Data Points: 6015
Starting Training
Done training
Test Results
Sensitivity: 0.5371710526315789
Specificity : 0.6904201680672268
Accuracy: 0.6129675810473816
ROC 0.613795610349
TP 1633 FP 921 TN 2054 FN 1407
None
Cross: Validation: [ 0.54388298 0.64145612 0.6263718 0.60708347 0.59062188]
Number of data points in benchmark 1196
Benchmark Results
Sensitivity: 0.6744186046511628
Specificity : 0.4822202948829141
Accuracy: 0.4891304347826087
ROC 0.578319449767
TP 29 FP 597 TN 556 FN 14
None
y pass Data/Benchmarks/phos_CK2.csv
Loading Data
Loaded Data
Working on Data
Sample Vector [13, 5, 10, 10, 12, 3, 20, 7, 12, 11, 13, 11, 13, 1.4692307692307693, 30.115384615384617, 0.07692307692307693]
Finished working with Data
Training Data Points: 66323
Test Data Points: 16581
Starting Training
Done training
Test Results
Sensitivity: 0.10714285714285714
Specificity : 0.9566433566433566
Accuracy: 0.8031481816537
ROC 0.531893106893
TP 321 FP 589 TN 12996 FN 2675
None
Cross: Validation: [ 0.80322036 0.80376312 0.80506634 0.8062123 0.80536791]
Number of data points in benchmark 719
Benchmark Results
Sensitivity: 0.2
Specificity : 0.9545454545454546
Accuracy: 0.9388038942976356
ROC 0.577272727273
TP 3 FP 32 TN 672 FN 12
None
x pass Data/Benchmarks/phos_CK2.csv
Loading Data
Loaded Data
Working on Data
Sample Vector [2, 17, 1, 2, 1, 6, 20, 16, 1, 17, 1, 4, 4, -0.6153846153846154, -22.623076923076926, 0.07692307692307693]
Finished working with Data
Training Data Points: 66323
Test Data Points: 16581
Starting Training
Done training
Test Results
Sensitivity: 0.09362808842652796
Specificity : 0.9603109959274343
Accuracy: 0.799529582051746
ROC 0.526969542177
TP 288 FP 536 TN 12969 FN 2788
None
Cross: Validation: [ 0.80430587 0.80316005 0.80428227 0.80392039 0.80609168]
Number of data points in benchmark 719
Benchmark Results
Sensitivity: 0.06666666666666667
Specificity : 0.9502840909090909
Accuracy: 0.9318497913769124
ROC 0.508475378788
TP 1 FP 35 TN 669 FN 14
None
y ADASYN Data/Benchmarks/phos_CK2.csv
Loading Data
Loaded Data
Working on Data
Sample Vector [1, 7, 10, 2, 19, 18, 20, 9, 12, 5, 10, 19, 7, -1.2076923076923078, -2.0769230769230753, 0.07692307692307693]
Balancing Data
Balanced Data
Finished working with Data
Training Data Points: 109781
Test Data Points: 27446
Starting Training
Done training
Test Results
Sensitivity: 0.7942745889818287
Specificity : 0.944468993960819
Accuracy: 0.8685782992057131
ROC 0.869371791471
TP 11015 FP 754 TN 12824 FN 2853
None
Cross: Validation: [ 0.52135102 0.91751075 0.94268537 0.94669339 0.94359628]
Number of data points in benchmark 719
Benchmark Results
Sensitivity: 0.4
Specificity : 0.9488636363636364
Accuracy: 0.9374130737134909
ROC 0.674431818182
TP 6 FP 36 TN 668 FN 9
None
x ADASYN Data/Benchmarks/phos_CK2.csv
Loading Data
Loaded Data
Working on Data
Sample Vector [3, 8, 7, 3, 7, 16, 20, 18, 17, 7, 8, 9, 12, -1.7153846153846155, 36.88461538461539, 0.0]
Balancing Data
Balanced Data
Finished working with Data
Training Data Points: 109773
Test Data Points: 27444
Starting Training
Done training
Test Results
Sensitivity: 0.7981763354394027
Specificity : 0.9437703462562889
Accuracy: 0.869880483894476
ROC 0.870973340848
TP 11117 FP 760 TN 12756 FN 2811
None
Cross: Validation: [ 0.51920274 0.91998251 0.94661662 0.94599716 0.94588784]
Number of data points in benchmark 719
Benchmark Results
Sensitivity: 0.13333333333333333
Specificity : 0.9417613636363636
Accuracy: 0.9248956884561892
ROC 0.537547348485
TP 2 FP 41 TN 663 FN 13
None
y SMOTEENN Data/Benchmarks/phos_CK2.csv
Loading Data
Loaded Data
Working on Data
Sample Vector [19, 11, 2, 9, 8, 3, 20, 2, 12, 8, 8, 2, 17, -0.8384615384615386, 53.67692307692308, 0.0]
Balancing Data
Balanced Data
Finished working with Data
Training Data Points: 102938
Test Data Points: 25735
Starting Training
Done training
Test Results
Sensitivity: 0.7987904543968617
Specificity : 0.961700866730869
Accuracy: 0.8842432484942685
ROC 0.880245660564
TP 9774 FP 517 TN 12982 FN 2462
None
Cross: Validation: [ 0.55910009 0.94847484 0.95119297 0.95550633 0.95130955]
Number of data points in benchmark 719
Benchmark Results
Sensitivity: 0.13333333333333333
Specificity : 0.9488636363636364
Accuracy: 0.9318497913769124
ROC 0.541098484848
TP 2 FP 36 TN 668 FN 13
None
x SMOTEENN Data/Benchmarks/phos_CK2.csv
Loading Data
Loaded Data
Working on Data
Sample Vector [8, 13, 2, 17, 10, 16, 20, 19, 12, 1, 3, 3, 9, -0.1538461538461539, 30.7, 0.0]
Balancing Data
Balanced Data
Finished working with Data
Training Data Points: 102939
Test Data Points: 25735
Starting Training
Done training
Test Results
Sensitivity: 0.8002615233736515
Specificity : 0.962071264538114
Accuracy: 0.8851369729939771
ROC 0.881166393956
TP 9792 FP 512 TN 12987 FN 2444
None
Cross: Validation: [ 0.56248057 0.94832142 0.95616694 0.95356338 0.95426284]
Number of data points in benchmark 719
Benchmark Results
Sensitivity: 0.26666666666666666
Specificity : 0.9360795454545454
Accuracy: 0.9221140472878998
ROC 0.601373106061
TP 4 FP 45 TN 659 FN 11
None
y random_under_sample Data/Benchmarks/phos_CK2.csv
Loading Data
Loaded Data
Working on Data
Sample Vector [5, 7, 3, 7, 5, 1, 20, 5, 10, 19, 9, 8, 8, -0.8846153846153846, 17.200000000000003, 0.23076923076923078]
Balancing Data
Balanced Data
Finished working with Data
Training Data Points: 24059
Test Data Points: 6015
Starting Training
Done training
Test Results
Sensitivity: 0.5447368421052632
Specificity : 0.6998319327731093
Accuracy: 0.6214463840399003
ROC 0.622284387439
TP 1656 FP 893 TN 2082 FN 1384
None
Cross: Validation: [ 0.61402926 0.61884973 0.61340206 0.61722647 0.6208846 ]
Number of data points in benchmark 719
Benchmark Results
Sensitivity: 0.6666666666666666
Specificity : 0.6107954545454546
Accuracy: 0.6119610570236439
ROC 0.638731060606
TP 10 FP 274 TN 430 FN 5
None
x random_under_sample Data/Benchmarks/phos_CK2.csv
Loading Data
Loaded Data
Working on Data
Sample Vector [11, 11, 13, 10, 5, 11, 20, 12, 9, 9, 9, 1, 7, -0.7384615384615384, 82.4846153846154, 0.07692307692307693]
Balancing Data
Balanced Data
Finished working with Data
Training Data Points: 24059
Test Data Points: 6015
Starting Training
Done training
Test Results
Sensitivity: 0.5463815789473684
Specificity : 0.6836974789915966
Accuracy: 0.6142975893599335
ROC 0.615039528969
TP 1661 FP 941 TN 2034 FN 1379
None
Cross: Validation: [ 0.63081782 0.61369681 0.62736947 0.61539741 0.62820086]
Number of data points in benchmark 719
Benchmark Results
Sensitivity: 0.7333333333333333
Specificity : 0.6107954545454546
Accuracy: 0.6133518776077886
ROC 0.672064393939
TP 11 FP 274 TN 430 FN 4
None
y ncl Data/Benchmarks/phos_CK2.csv
Loading Data
Loaded Data
Working on Data
Sample Vector [1, 8, 9, 11, 9, 2, 11, 11, 11, 2, 20, 11, 8, -1.4999999999999998, 126.30769230769232, 0.0]
Balancing Data
Balanced Data
Finished working with Data
Training Data Points: 47220
Test Data Points: 11806
Starting Training
Done training
Test Results
Sensitivity: 0.23407155025553664
Specificity : 0.9319129748619096
Accuracy: 0.7584279180077926
ROC 0.582992262559
TP 687 FP 604 TN 8267 FN 2248
None
Cross: Validation: [ 0.75749619 0.75876673 0.75933926 0.75442609 0.75559132]
Number of data points in benchmark 719
Benchmark Results
Sensitivity: 0.3333333333333333
Specificity : 0.8821022727272727
Accuracy: 0.8706536856745479
ROC 0.60771780303
TP 5 FP 83 TN 621 FN 10
None
x ncl Data/Benchmarks/phos_CK2.csv
Loading Data
Loaded Data
Working on Data
Sample Vector [13, 8, 9, 3, 19, 10, 20, 20, 19, 9, 19, 9, 13, -1.0692307692307692, 68.4923076923077, 0.0]
Balancing Data
Balanced Data
Finished working with Data
Training Data Points: 47245
Test Data Points: 11812
Starting Training
Done training
Test Results
Sensitivity: 0.23575571477311497
Specificity : 0.9313140412115752
Accuracy: 0.7587199458178124
ROC 0.583534877992
TP 691 FP 610 TN 8271 FN 2240
None
Cross: Validation: [ 0.74771419 0.75736539 0.75742951 0.75302684 0.76377953]
Number of data points in benchmark 719
Benchmark Results
Sensitivity: 0.2
Specificity : 0.8877840909090909
Accuracy: 0.8734353268428373
ROC 0.543892045455
TP 3 FP 79 TN 625 FN 12
None
y near_miss Data/Benchmarks/phos_CK2.csv
Loading Data
Loaded Data
Working on Data
Sample Vector [12, 12, 11, 2, 18, 20, 20, 10, 2, 13, 3, 10, 1, 0.9076923076923076, 47.14615384615385, 0.0]
Balancing Data
Balanced Data
Finished working with Data
Training Data Points: 24059
Test Data Points: 6015
Starting Training
Done training
Test Results
Sensitivity: 0.5592105263157895
Specificity : 0.6840336134453782
Accuracy: 0.6209476309226932
ROC 0.621622069881
TP 1700 FP 940 TN 2035 FN 1340
None
Cross: Validation: [ 0.55119681 0.64577793 0.63219155 0.61639508 0.59810442]
Number of data points in benchmark 719
Benchmark Results
Sensitivity: 0.6666666666666666
Specificity : 0.43607954545454547
Accuracy: 0.44089012517385257
ROC 0.551373106061
TP 10 FP 397 TN 307 FN 5
None
x near_miss Data/Benchmarks/phos_CK2.csv
Loading Data
Loaded Data
Working on Data
Sample Vector [16, 17, 8, 5, 9, 17, 20, 13, 8, 20, 11, 13, 1, -1.0999999999999999, 18.24615384615385, 0.07692307692307693]
Balancing Data
Balanced Data
Finished working with Data
Training Data Points: 24059
Test Data Points: 6015
Starting Training
Done training
Test Results
Sensitivity: 0.5546052631578947
Specificity : 0.6783193277310925
Accuracy: 0.6157938487115544
ROC 0.616462295444
TP 1686 FP 957 TN 2018 FN 1354
None
Cross: Validation: [ 0.53989362 0.63979388 0.62653808 0.61589624 0.58995677]
Number of data points in benchmark 719
Benchmark Results
Sensitivity: 0.6
Specificity : 0.46732954545454547
Accuracy: 0.47009735744089015
ROC 0.533664772727
TP 9 FP 375 TN 329 FN 6
None
y pass Data/Benchmarks/phos_MAPK1.csv
Loading Data
Loaded Data
Working on Data
Sample Vector [10, 8, 20, 20, 3, 11, 20, 3, 1, 12, 2, 12, 18, 0.4538461538461539, 38.861538461538466, 0.0]
Finished working with Data
Training Data Points: 66323
Test Data Points: 16581
Starting Training
Done training
Test Results
Sensitivity: 0.11637790127492645
Specificity : 0.9617660109451265
Accuracy: 0.8058018213617997
ROC 0.53907195611
TP 356 FP 517 TN 13005 FN 2703
None
Cross: Validation: [ 0.80394404 0.80394404 0.80784077 0.80277443 0.80428227]
Number of data points in benchmark 421
Benchmark Results
Sensitivity: 0.19230769230769232
Specificity : 0.9341772151898734
Accuracy: 0.8883610451306413
ROC 0.563242453749
TP 5 FP 26 TN 369 FN 21
None
x pass Data/Benchmarks/phos_MAPK1.csv
Loading Data
Loaded Data
Working on Data
Sample Vector [11, 3, 16, 4, 3, 2, 20, 16, 2, 10, 2, 10, 2, 0.4923076923076922, 9.230769230769232, 0.0]
Finished working with Data
Training Data Points: 66323
Test Data Points: 16581
Starting Training
Done training
Test Results
Sensitivity: 0.11321392362284556
Specificity : 0.958596388195566
Accuracy: 0.8077317411495085
ROC 0.535905155909
TP 335 FP 564 TN 13058 FN 2624
None
Cross: Validation: [ 0.80533108 0.8045471 0.80518697 0.8048854 0.80494572]
Number of data points in benchmark 421
Benchmark Results
Sensitivity: 0.19230769230769232
Specificity : 0.9392405063291139
Accuracy: 0.8931116389548693
ROC 0.565774099318
TP 5 FP 24 TN 371 FN 21
None
y ADASYN Data/Benchmarks/phos_MAPK1.csv
Loading Data
Loaded Data
Working on Data
Sample Vector [1, 8, 17, 3, 19, 9, 20, 12, 2, 15, 9, 8, 7, -1.4230769230769231, 18.24615384615385, 0.07692307692307693]
Balancing Data
Balanced Data
Finished working with Data
Training Data Points: 109761
Test Data Points: 27441
Starting Training
Done training
Test Results
Sensitivity: 0.7946140035906643
Specificity : 0.9465078425569695
Accuracy: 0.8694289566706752
ROC 0.870560923074
TP 11065 FP 723 TN 12793 FN 2860
None
Cross: Validation: [ 0.52086294 0.91815167 0.94573615 0.94260204 0.94500729]
Number of data points in benchmark 421
Benchmark Results
Sensitivity: 0.3076923076923077
Specificity : 0.9215189873417722
Accuracy: 0.8836104513064132
ROC 0.614605647517
TP 8 FP 31 TN 364 FN 18
None
x ADASYN Data/Benchmarks/phos_MAPK1.csv
Loading Data
Loaded Data
Working on Data
Sample Vector [9, 20, 12, 19, 9, 3, 20, 1, 7, 18, 18, 8, 13, -1.0923076923076924, -21.19230769230769, 0.0]
Balancing Data
Balanced Data
Finished working with Data
Training Data Points: 109785
Test Data Points: 27447
Starting Training
Done training
Test Results
Sensitivity: 0.7983223660423747
Specificity : 0.9457335878983698
Accuracy: 0.8714613618974751
ROC 0.87202797697
TP 11040 FP 739 TN 12879 FN 2789
None
Cross: Validation: [ 0.51943746 0.91758662 0.94152153 0.94866283 0.94516505]
Number of data points in benchmark 421
Benchmark Results
Sensitivity: 0.38461538461538464
Specificity : 0.9291139240506329
Accuracy: 0.8954869358669834
ROC 0.656864654333
TP 10 FP 28 TN 367 FN 16
None
y SMOTEENN Data/Benchmarks/phos_MAPK1.csv
Loading Data
Loaded Data
Working on Data
Sample Vector [18, 16, 9, 13, 9, 3, 20, 7, 12, 16, 12, 9, 17, -0.9846153846153847, 51.31538461538462, 0.0]
Balancing Data
Balanced Data
Finished working with Data
Training Data Points: 103008
Test Data Points: 25752
Starting Training
Done training
Test Results
Sensitivity: 0.804033640891647
Specificity : 0.9605331358756016
Accuracy: 0.8861059335197267
ROC 0.882283388384
TP 9847 FP 533 TN 12972 FN 2400
None
Cross: Validation: [ 0.5590805 0.94513261 0.95394532 0.95522504 0.95429304]
Number of data points in benchmark 421
Benchmark Results
Sensitivity: 0.19230769230769232
Specificity : 0.9367088607594937
Accuracy: 0.8907363420427553
ROC 0.564508276534
TP 5 FP 25 TN 370 FN 21
None
x SMOTEENN Data/Benchmarks/phos_MAPK1.csv
Loading Data
Loaded Data
Working on Data
Sample Vector [15, 9, 16, 7, 20, 20, 20, 20, 10, 18, 14, 18, 6, -1.6076923076923075, 5.653846153846157, 0.15384615384615385]
Balancing Data
Balanced Data
Finished working with Data
Training Data Points: 102873
Test Data Points: 25719
Starting Training
Done training
Test Results
Sensitivity: 0.8036754450734268
Specificity : 0.9584626755358463
Accuracy: 0.88510439752712
ROC 0.881069060305
TP 9796 FP 562 TN 12968 FN 2393
None
Cross: Validation: [ 0.56102492 0.94980365 0.9569951 0.95334007 0.95617855]
Number of data points in benchmark 421
Benchmark Results
Sensitivity: 0.19230769230769232
Specificity : 0.9265822784810127
Accuracy: 0.8812351543942993
ROC 0.559444985394
TP 5 FP 29 TN 366 FN 21
None
y random_under_sample Data/Benchmarks/phos_MAPK1.csv
Loading Data
Loaded Data
Working on Data
Sample Vector [9, 9, 5, 11, 19, 6, 20, 20, 17, 12, 7, 8, 13, -1.1384615384615389, 47.73076923076923, 0.15384615384615385]
Balancing Data
Balanced Data
Finished working with Data
Training Data Points: 24059
Test Data Points: 6015
Starting Training
Done training
Test Results
Sensitivity: 0.5194078947368421
Specificity : 0.7011764705882353
Accuracy: 0.6093100581878637
ROC 0.610292182663
TP 1579 FP 889 TN 2086 FN 1461
None
Cross: Validation: [ 0.62200798 0.61386303 0.62221483 0.62254739 0.61739275]
Number of data points in benchmark 421
Benchmark Results
Sensitivity: 0.5769230769230769
Specificity : 0.6126582278481013
Accuracy: 0.6104513064133017
ROC 0.594790652386
TP 15 FP 153 TN 242 FN 11
None
x random_under_sample Data/Benchmarks/phos_MAPK1.csv
Loading Data
Loaded Data
Working on Data
Sample Vector [13, 16, 10, 13, 11, 10, 20, 10, 12, 10, 20, 1, 10, 0.19999999999999998, 24.79230769230769, 0.0]
Balancing Data
Balanced Data
Finished working with Data
Training Data Points: 24059
Test Data Points: 6015
Starting Training
Done training
Test Results
Sensitivity: 0.5542763157894737
Specificity : 0.7129411764705882
Accuracy: 0.6327514546965919
ROC 0.63360874613
TP 1685 FP 854 TN 2121 FN 1355
None
Cross: Validation: [ 0.61585771 0.61768617 0.63451945 0.63269039 0.62554041]
Number of data points in benchmark 421
Benchmark Results
Sensitivity: 0.6923076923076923
Specificity : 0.620253164556962
Accuracy: 0.6247030878859857
ROC 0.656280428432
TP 18 FP 150 TN 245 FN 8
None
y ncl Data/Benchmarks/phos_MAPK1.csv
Loading Data
Loaded Data
Working on Data
Sample Vector [14, 20, 12, 3, 7, 11, 20, 3, 4, 2, 2, 12, 11, 1.1923076923076925, 46.284615384615385, 0.0]
Balancing Data
Balanced Data
Finished working with Data
Training Data Points: 47196
Test Data Points: 11799
Starting Training
Done training
Test Results
Sensitivity: 0.24380305602716468
Specificity : 0.9327987350350124
Accuracy: 0.7608271887448089
ROC 0.588300895531
TP 718 FP 595 TN 8259 FN 2227
None
Cross: Validation: [ 0.75364407 0.75669492 0.75286041 0.75343279 0.75970503]
Number of data points in benchmark 421
Benchmark Results
Sensitivity: 0.46153846153846156
Specificity : 0.8810126582278481
Accuracy: 0.8551068883610451
ROC 0.671275559883
TP 12 FP 47 TN 348 FN 14
None
x ncl Data/Benchmarks/phos_MAPK1.csv
Loading Data
Loaded Data
Working on Data
Sample Vector [7, 12, 3, 2, 1, 9, 20, 3, 10, 12, 18, 19, 11, -0.007692307692307699, -3.100000000000001, 0.0]
Balancing Data
Balanced Data
Finished working with Data
Training Data Points: 47189
Test Data Points: 11798
Starting Training
Done training
Test Results
Sensitivity: 0.23087385243114586
Specificity : 0.9356441232923112
Accuracy: 0.7599593151381591
ROC 0.583258987862
TP 679 FP 570 TN 8287 FN 2262
None
Cross: Validation: [ 0.75504323 0.75809459 0.75620921 0.75349665 0.75553107]
Number of data points in benchmark 421
Benchmark Results
Sensitivity: 0.34615384615384615
Specificity : 0.8556962025316456
Accuracy: 0.8242280285035629
ROC 0.600925024343
TP 9 FP 57 TN 338 FN 17
None
y near_miss Data/Benchmarks/phos_MAPK1.csv
Loading Data
Loaded Data
Working on Data
Sample Vector [7, 16, 20, 3, 2, 7, 8, 7, 1, 1, 20, 19, 0, -1.5416666666666667, 18.71666666666667, 0.0]
Balancing Data
Balanced Data
Finished working with Data
Training Data Points: 24059
Test Data Points: 6015
Starting Training
Done training
Test Results
Sensitivity: 0.5319078947368421
Specificity : 0.6773109243697479
Accuracy: 0.6038237738985869
ROC 0.604609409553
TP 1617 FP 960 TN 2015 FN 1423
None
Cross: Validation: [ 0.54255319 0.64611037 0.62736947 0.61323578 0.59627536]
Number of data points in benchmark 421
Benchmark Results
Sensitivity: 0.5769230769230769
Specificity : 0.5189873417721519
Accuracy: 0.5225653206650831
ROC 0.547955209348
TP 15 FP 190 TN 205 FN 11
None
x near_miss Data/Benchmarks/phos_MAPK1.csv
Loading Data
Loaded Data
Working on Data
Sample Vector [18, 1, 20, 14, 13, 5, 20, 10, 20, 19, 2, 16, 17, -0.49230769230769234, -15.392307692307691, 0.07692307692307693]
Balancing Data
Balanced Data
Finished working with Data
Training Data Points: 24059
Test Data Points: 6015
Starting Training
Done training
Test Results
Sensitivity: 0.5516447368421052
Specificity : 0.6833613445378152
Accuracy: 0.6167913549459684
ROC 0.61750304069
TP 1677 FP 942 TN 2033 FN 1363
None
Cross: Validation: [ 0.54138963 0.6484375 0.62221483 0.61423346 0.58280678]
Number of data points in benchmark 421
Benchmark Results
Sensitivity: 0.6538461538461539
Specificity : 0.48860759493670886
Accuracy: 0.498812351543943
ROC 0.571226874391
TP 17 FP 202 TN 193 FN 9
None
y pass Data/Benchmarks/phos_PKA.csv
Loading Data
Loaded Data
Working on Data
Sample Vector [8, 20, 17, 3, 9, 9, 20, 8, 2, 3, 3, 0, 0, -0.6090909090909088, 31.872727272727275, 0.0]
Finished working with Data
Training Data Points: 66323
Test Data Points: 16581
Starting Training
Done training
Test Results
Sensitivity: 0.1054713249835201
Specificity : 0.9613198494131542
Accuracy: 0.8047162414812135
ROC 0.533395587198
TP 320 FP 524 TN 13023 FN 2714
None
Cross: Validation: [ 0.805512 0.8035822 0.80518697 0.80536791 0.80560917]
Number of data points in benchmark 2130
Benchmark Results
Sensitivity: 0.41379310344827586
Specificity : 0.9685863874345549
Accuracy: 0.9610328638497653
ROC 0.691189745441
TP 12 FP 66 TN 2035 FN 17
None
x pass Data/Benchmarks/phos_PKA.csv
Loading Data
Loaded Data
Working on Data
Sample Vector [17, 11, 2, 9, 11, 12, 20, 11, 17, 17, 20, 17, 3, -1.376923076923077, 100.22307692307695, 0.0]
Finished working with Data
Training Data Points: 66323
Test Data Points: 16581
Starting Training
Done training
Test Results
Sensitivity: 0.11132749107432652
Specificity : 0.9584444444444444
Accuracy: 0.8010373318858935
ROC 0.534885967759
TP 343 FP 561 TN 12939 FN 2738
None
Cross: Validation: [ 0.80388373 0.80400434 0.80470446 0.80271411 0.80633293]
Number of data points in benchmark 2130
Benchmark Results
Sensitivity: 0.4827586206896552
Specificity : 0.9652546406473108
Accuracy: 0.9586854460093897
ROC 0.724006630668
TP 14 FP 73 TN 2028 FN 15
None
y ADASYN Data/Benchmarks/phos_PKA.csv
Loading Data
Loaded Data
Working on Data
Sample Vector [7, 20, 1, 3, 13, 20, 20, 20, 10, 17, 7, 3, 19, -0.5923076923076923, 10.984615384615388, 0.0]
Balancing Data
Balanced Data
Finished working with Data
Training Data Points: 109788
Test Data Points: 27447
Starting Training
Done training
Test Results
Sensitivity: 0.7954790096878364
Specificity : 0.9455298993487271
Accuracy: 0.869348198345903
ROC 0.870504454518
TP 11085 FP 736 TN 12776 FN 2850
None
Cross: Validation: [ 0.52014719 0.91948412 0.94400117 0.94793413 0.9455294 ]
Number of data points in benchmark 2130
Benchmark Results
Sensitivity: 0.41379310344827586
Specificity : 0.9581151832460733
Accuracy: 0.9507042253521126
ROC 0.685954143347
TP 12 FP 88 TN 2013 FN 17
None
x ADASYN Data/Benchmarks/phos_PKA.csv
Loading Data
Loaded Data
Working on Data
Sample Vector [3, 13, 8, 11, 12, 19, 20, 13, 10, 13, 10, 10, 3, 1.046153846153846, 30.553846153846155, 0.0]
Balancing Data
Balanced Data
Finished working with Data
Training Data Points: 109777
Test Data Points: 27445
Starting Training
Done training
Test Results
Sensitivity: 0.7897543050652064
Specificity : 0.9466312840925845
Accuracy: 0.867298232829295
ROC 0.868192794579
TP 10961 FP 724 TN 12842 FN 2918
None
Cross: Validation: [ 0.51794498 0.91805429 0.94760239 0.9450153 0.94465093]
Number of data points in benchmark 2130
Benchmark Results
Sensitivity: 0.5517241379310345
Specificity : 0.9500237981913374
Accuracy: 0.9446009389671362
ROC 0.750873968061
TP 16 FP 105 TN 1996 FN 13
None
y SMOTEENN Data/Benchmarks/phos_PKA.csv
Loading Data
Loaded Data
Working on Data
Sample Vector [1, 7, 16, 18, 9, 3, 20, 12, 13, 13, 19, 1, 18, -0.4307692307692306, 14.384615384615381, 0.0]
Balancing Data
Balanced Data
Finished working with Data
Training Data Points: 102941
Test Data Points: 25736
Starting Training
Done training
Test Results
Sensitivity: 0.7989152765223108
Specificity : 0.9567332497973022
Accuracy: 0.8821106621075536
ROC 0.87782426316
TP 9722 FP 587 TN 12980 FN 2447
None
Cross: Validation: [ 0.55878924 0.947894 0.95360404 0.95461434 0.95679036]
Number of data points in benchmark 2130
Benchmark Results
Sensitivity: 0.4482758620689655
Specificity : 0.9609709662065683
Accuracy: 0.9539906103286385
ROC 0.704623414138
TP 13 FP 82 TN 2019 FN 16
None
x SMOTEENN Data/Benchmarks/phos_PKA.csv
Loading Data
Loaded Data
Working on Data
Sample Vector [15, 16, 7, 17, 19, 5, 20, 9, 12, 3, 10, 11, 18, -1.2076923076923074, 104.83076923076923, 0.15384615384615385]
Balancing Data
Balanced Data
Finished working with Data
Training Data Points: 103017
Test Data Points: 25755
Starting Training
Done training
Test Results
Sensitivity: 0.8038020960272971
Specificity : 0.9590212702662502
Accuracy: 0.8848378955542613
ROC 0.881411683147
TP 9894 FP 551 TN 12895 FN 2415
None
Cross: Validation: [ 0.55993011 0.94703941 0.9524734 0.95297818 0.95573503]
Number of data points in benchmark 2130
Benchmark Results
Sensitivity: 0.27586206896551724
Specificity : 0.9619228938600667
Accuracy: 0.9525821596244132
ROC 0.618892481413
TP 8 FP 80 TN 2021 FN 21
None
y random_under_sample Data/Benchmarks/phos_PKA.csv
Loading Data
Loaded Data
Working on Data
Sample Vector [16, 7, 19, 9, 1, 11, 20, 2, 2, 2, 2, 11, 9, -1.1307692307692305, 55.63076923076923, 0.0]
Balancing Data
Balanced Data
Finished working with Data
Training Data Points: 24059
Test Data Points: 6015
Starting Training
Done training
Test Results
Sensitivity: 0.5355263157894737
Specificity : 0.7216806722689075
Accuracy: 0.6275976724854531
ROC 0.628603494029
TP 1628 FP 828 TN 2147 FN 1412
None
Cross: Validation: [ 0.63347739 0.62150931 0.6208846 0.61506485 0.61489857]
Number of data points in benchmark 2130
Benchmark Results
Sensitivity: 0.7931034482758621
Specificity : 0.6687291765825797
Accuracy: 0.6704225352112676
ROC 0.730916312429
TP 23 FP 696 TN 1405 FN 6
None
x random_under_sample Data/Benchmarks/phos_PKA.csv
Loading Data
Loaded Data
Working on Data
Sample Vector [9, 14, 9, 19, 9, 12, 20, 10, 19, 11, 2, 20, 19, -1.2538461538461536, 66.13076923076923, 0.0]
Balancing Data
Balanced Data
Finished working with Data
Training Data Points: 24059
Test Data Points: 6015
Starting Training
Done training
Test Results
Sensitivity: 0.5345394736842105
Specificity : 0.6954621848739496
Accuracy: 0.6141313383208645
ROC 0.615000829279
TP 1625 FP 906 TN 2069 FN 1415
None
Cross: Validation: [ 0.62549867 0.62383644 0.60774859 0.61290323 0.62105088]
Number of data points in benchmark 2130
Benchmark Results
Sensitivity: 0.6551724137931034
Specificity : 0.6692051404093289
Accuracy: 0.6690140845070423
ROC 0.662188777101
TP 19 FP 695 TN 1406 FN 10
None
y ncl Data/Benchmarks/phos_PKA.csv
Loading Data
Loaded Data
Working on Data
Sample Vector [9, 11, 9, 18, 2, 12, 20, 13, 20, 11, 3, 17, 11, -0.5307692307692308, 60.51538461538462, 0.0]
Balancing Data
Balanced Data
Finished working with Data
Training Data Points: 47245
Test Data Points: 11812
Starting Training
Done training
Test Results
Sensitivity: 0.25247355851245307
Specificity : 0.9289494426303344
Accuracy: 0.7610904165255672
ROC 0.590711500571
TP 740 FP 631 TN 8250 FN 2191
None
Cross: Validation: [ 0.75228581 0.75702675 0.76293286 0.75328084 0.75827618]
Number of data points in benchmark 2130
Benchmark Results
Sensitivity: 0.5172413793103449
Specificity : 0.9224178962398858
Accuracy: 0.9169014084507042
ROC 0.719829637775
TP 15 FP 163 TN 1938 FN 14
None
x ncl Data/Benchmarks/phos_PKA.csv
Loading Data
Loaded Data
Working on Data
Sample Vector [5, 1, 3, 2, 17, 8, 20, 19, 10, 9, 4, 20, 1, -0.5923076923076923, 44.930769230769236, 0.07692307692307693]
Balancing Data
Balanced Data
Finished working with Data
Training Data Points: 47233
Test Data Points: 11809
Starting Training
Done training
Test Results
Sensitivity: 0.23673469387755103
Specificity : 0.9351674371406021
Accuracy: 0.7612837666186807
ROC 0.585951065509
TP 696 FP 575 TN 8294 FN 2244
None
Cross: Validation: [ 0.75628758 0.76060632 0.75533537 0.75838415 0.75550474]
Number of data points in benchmark 2130
Benchmark Results
Sensitivity: 0.5172413793103449
Specificity : 0.9243217515468825
Accuracy: 0.9187793427230047
ROC 0.720781565429
TP 15 FP 159 TN 1942 FN 14
None
y near_miss Data/Benchmarks/phos_PKA.csv
Loading Data
Loaded Data
Working on Data
Sample Vector [2, 20, 10, 1, 1, 10, 20, 2, 2, 10, 20, 2, 10, 0.08461538461538462, 18.723076923076924, 0.0]
Balancing Data
Balanced Data
Finished working with Data
Training Data Points: 24059
Test Data Points: 6015
Starting Training
Done training
Test Results
Sensitivity: 0.5519736842105263
Specificity : 0.6823529411764706
Accuracy: 0.6164588528678304
ROC 0.617163312693
TP 1678 FP 945 TN 2030 FN 1362
None
Cross: Validation: [ 0.54005984 0.64328457 0.62038577 0.61523113 0.60392418]
Number of data points in benchmark 2130
Benchmark Results
Sensitivity: 0.6896551724137931
Specificity : 0.5249881009043312
Accuracy: 0.5272300469483568
ROC 0.607321636659
TP 20 FP 998 TN 1103 FN 9
None
x near_miss Data/Benchmarks/phos_PKA.csv
Loading Data
Loaded Data
Working on Data
Sample Vector [10, 19, 9, 10, 6, 9, 20, 3, 11, 1, 7, 19, 9, -1.7538461538461536, 32.330769230769235, 0.07692307692307693]
Balancing Data
Balanced Data
Finished working with Data
Training Data Points: 24059
Test Data Points: 6015
Starting Training
Done training
Test Results
Sensitivity: 0.537828947368421
Specificity : 0.6756302521008404
Accuracy: 0.6059850374064838
ROC 0.606729599735
TP 1635 FP 965 TN 2010 FN 1405
None
Cross: Validation: [ 0.54255319 0.64295213 0.62454273 0.60758231 0.58829398]
Number of data points in benchmark 2130
Benchmark Results
Sensitivity: 0.8275862068965517
Specificity : 0.5069014754878629
Accuracy: 0.5112676056338028
ROC 0.667243841192
TP 24 FP 1036 TN 1065 FN 5
None
y pass Data/Benchmarks/phos_PKC.csv
Loading Data
Loaded Data
Working on Data
Sample Vector [17, 17, 1, 2, 4, 17, 20, 19, 2, 19, 7, 18, 15, -1.9076923076923078, 21.369230769230764, 0.07692307692307693]
Finished working with Data
Training Data Points: 66323
Test Data Points: 16581
Starting Training
Done training
Test Results
Sensitivity: 0.11610738255033556
Specificity : 0.9597088449378722
Accuracy: 0.8080936011097039
ROC 0.537908113744
TP 346 FP 548 TN 13053 FN 2634
None
Cross: Validation: [ 0.80309975 0.80617537 0.80476478 0.80193004 0.80603136]
Number of data points in benchmark 1064
Benchmark Results
Sensitivity: 0.21428571428571427
Specificity : 0.9584942084942085
Accuracy: 0.9389097744360902
ROC 0.58638996139
TP 6 FP 43 TN 993 FN 22
None
x pass Data/Benchmarks/phos_PKC.csv
Loading Data
Loaded Data
Working on Data
Sample Vector [2, 10, 11, 16, 10, 12, 20, 19, 9, 19, 9, 1, 6, -1.2615384615384617, 60.80769230769231, 0.07692307692307693]
Finished working with Data
Training Data Points: 66323
Test Data Points: 16581
Starting Training
Done training
Test Results
Sensitivity: 0.1105476673427992
Specificity : 0.9594068854143728
Accuracy: 0.8079729811229721
ROC 0.534977276379
TP 327 FP 553 TN 13070 FN 2631
None
Cross: Validation: [ 0.80448679 0.80502955 0.80482509 0.80566948 0.80603136]
Number of data points in benchmark 1064
Benchmark Results
Sensitivity: 0.17857142857142858
Specificity : 0.9584942084942085
Accuracy: 0.9379699248120301
ROC 0.568532818533
TP 5 FP 43 TN 993 FN 23
None
y ADASYN Data/Benchmarks/phos_PKC.csv
Loading Data
Loaded Data
Working on Data
Sample Vector [12, 7, 3, 3, 9, 5, 20, 3, 9, 8, 18, 5, 9, -0.06923076923076933, 3.7384615384615394, 0.15384615384615385]
Balancing Data
Balanced Data
Finished working with Data
Training Data Points: 109772
Test Data Points: 27443
Starting Training
Done training
Test Results
Sensitivity: 0.799796378445204
Specificity : 0.9434706397896582
Accuracy: 0.8714790656998141
ROC 0.871633509117
TP 10998 FP 774 TN 12918 FN 2753
None
Cross: Validation: [ 0.51829179 0.91688529 0.94384725 0.94417317 0.9465418 ]
Number of data points in benchmark 1064
Benchmark Results
Sensitivity: 0.07142857142857142
Specificity : 0.9488416988416989
Accuracy: 0.9257518796992481
ROC 0.510135135135
TP 2 FP 53 TN 983 FN 26
None
x ADASYN Data/Benchmarks/phos_PKC.csv
Loading Data
Loaded Data
Working on Data
Sample Vector [20, 1, 18, 1, 1, 10, 20, 13, 1, 10, 10, 11, 20, -0.5153846153846154, 42.71538461538462, 0.0]
Balancing Data
Balanced Data
Finished working with Data
Training Data Points: 109790
Test Data Points: 27448
Starting Training
Done training
Test Results
Sensitivity: 0.796872748234616
Specificity : 0.9447310243183493
Accuracy: 0.8699723112795104
ROC 0.870801886276
TP 11059 FP 750 TN 12820 FN 2819
None
Cross: Validation: [ 0.51976393 0.9170067 0.9486647 0.9439283 0.94811819]
Number of data points in benchmark 1064
Benchmark Results
Sensitivity: 0.14285714285714285
Specificity : 0.9411196911196911
Accuracy: 0.9201127819548872
ROC 0.541988416988
TP 4 FP 61 TN 975 FN 24
None
y SMOTEENN Data/Benchmarks/phos_PKC.csv
Loading Data
Loaded Data
Working on Data
Sample Vector [20, 10, 7, 3, 9, 7, 20, 12, 7, 12, 20, 2, 7, -0.6153846153846152, -12.576923076923078, 0.0]
Balancing Data
Balanced Data
Finished working with Data
Training Data Points: 102957
Test Data Points: 25740
Starting Training
Done training
Test Results
Sensitivity: 0.7999836721365009
Specificity : 0.9603439329923653
Accuracy: 0.884032634032634
ROC 0.880163802564
TP 9799 FP 535 TN 12956 FN 2450
None
Cross: Validation: [ 0.56087801 0.94417249 0.95376666 0.95240685 0.95504876]
Number of data points in benchmark 1064
Benchmark Results
Sensitivity: 0.10714285714285714
Specificity : 0.9575289575289575
Accuracy: 0.9351503759398496
ROC 0.532335907336
TP 3 FP 44 TN 992 FN 25
None
x SMOTEENN Data/Benchmarks/phos_PKC.csv
Loading Data
Loaded Data
Working on Data
Sample Vector [17, 17, 19, 10, 19, 3, 20, 14, 3, 8, 7, 3, 3, -0.5538461538461539, 101.46923076923078, 0.0]
Balancing Data
Balanced Data
Finished working with Data
Training Data Points: 103008
Test Data Points: 25752
Starting Training
Done training
Test Results
Sensitivity: 0.8072180942271576
Specificity : 0.9608293224731581
Accuracy: 0.887775706741224
ROC 0.88402370835
TP 9886 FP 529 TN 12976 FN 2361
None
Cross: Validation: [ 0.56210927 0.94664699 0.95437248 0.9544872 0.9539047 ]
Number of data points in benchmark 1064
Benchmark Results
Sensitivity: 0.17857142857142858
Specificity : 0.9575289575289575
Accuracy: 0.9370300751879699
ROC 0.56805019305
TP 5 FP 44 TN 992 FN 23
None
y random_under_sample Data/Benchmarks/phos_PKC.csv
Loading Data
Loaded Data
Working on Data
Sample Vector [3, 16, 15, 4, 12, 8, 20, 10, 20, 5, 17, 2, 9, -0.2846153846153847, 86.32307692307693, 0.15384615384615385]
Balancing Data
Balanced Data
Finished working with Data
Training Data Points: 24059
Test Data Points: 6015
Starting Training
Done training
Test Results
Sensitivity: 0.5470394736842106
Specificity : 0.7011764705882353
Accuracy: 0.6232751454696592
ROC 0.624107972136
TP 1663 FP 889 TN 2086 FN 1377
None
Cross: Validation: [ 0.62134309 0.62849069 0.61789159 0.61489857 0.62204855]
Number of data points in benchmark 1064
Benchmark Results
Sensitivity: 0.5714285714285714
Specificity : 0.6486486486486487
Accuracy: 0.6466165413533834
ROC 0.610038610039
TP 16 FP 364 TN 672 FN 12
None
x random_under_sample Data/Benchmarks/phos_PKC.csv
Loading Data
Loaded Data
Working on Data
Sample Vector [3, 7, 10, 7, 18, 1, 20, 12, 20, 9, 7, 20, 9, -1.3461538461538463, 7.7076923076923105, 0.0]
Balancing Data
Balanced Data
Finished working with Data
Training Data Points: 24059
Test Data Points: 6015
Starting Training
Done training
Test Results
Sensitivity: 0.5417763157894737
Specificity : 0.706218487394958
Accuracy: 0.6231088944305901
ROC 0.623997401592
TP 1647 FP 874 TN 2101 FN 1393
None
Cross: Validation: [ 0.61818484 0.6221742 0.61905554 0.62919854 0.62903226]
Number of data points in benchmark 1064
Benchmark Results
Sensitivity: 0.5714285714285714
Specificity : 0.6418918918918919
Accuracy: 0.6400375939849624
ROC 0.60666023166
TP 16 FP 371 TN 665 FN 12
None
y ncl Data/Benchmarks/phos_PKC.csv
Loading Data
Loaded Data
Working on Data
Sample Vector [4, 20, 20, 20, 12, 10, 20, 1, 2, 20, 2, 0, 0, 0.45454545454545453, -8.963636363636363, 0.0]
Balancing Data
Balanced Data
Finished working with Data
Training Data Points: 47246
Test Data Points: 11812
Starting Training
Done training
Test Results
Sensitivity: 0.2436028659160696
Specificity : 0.9296250422249747
Accuracy: 0.7593972231628852
ROC 0.586613954071
TP 714 FP 625 TN 8256 FN 2217
None
Cross: Validation: [ 0.75738593 0.7556722 0.76191686 0.75675218 0.75285751]
Number of data points in benchmark 1064
Benchmark Results
Sensitivity: 0.32142857142857145
Specificity : 0.8976833976833977
Accuracy: 0.8825187969924813
ROC 0.609555984556
TP 9 FP 106 TN 930 FN 19
None
x ncl Data/Benchmarks/phos_PKC.csv
Loading Data
Loaded Data
Working on Data
Sample Vector [15, 5, 3, 4, 20, 9, 11, 11, 3, 18, 20, 0, 0, -0.054545454545454654, 51.28181818181818, 0.18181818181818182]
Balancing Data
Balanced Data
Finished working with Data
Training Data Points: 47264
Test Data Points: 11816
Starting Training
Done training
Test Results
Sensitivity: 0.240734444066644
Specificity : 0.932056338028169
Accuracy: 0.7599864590385917
ROC 0.586395391047
TP 708 FP 603 TN 8272 FN 2233
None
Cross: Validation: [ 0.75569095 0.75806042 0.75964794 0.75573424 0.75319509]
Number of data points in benchmark 1064
Benchmark Results
Sensitivity: 0.2857142857142857
Specificity : 0.9131274131274131
Accuracy: 0.8966165413533834
ROC 0.599420849421
TP 8 FP 90 TN 946 FN 20
None
y near_miss Data/Benchmarks/phos_PKC.csv
Loading Data
Loaded Data
Working on Data
Sample Vector [19, 14, 5, 9, 17, 17, 20, 7, 17, 18, 17, 18, 17, -2.753846153846154, 97.35384615384616, 0.07692307692307693]
Balancing Data
Balanced Data
Finished working with Data
Training Data Points: 24059
Test Data Points: 6015
Starting Training
Done training
Test Results
Sensitivity: 0.5444078947368421
Specificity : 0.6732773109243697
Accuracy: 0.6081463009143807
ROC 0.608842602831
TP 1655 FP 972 TN 2003 FN 1385
None
Cross: Validation: [ 0.54138963 0.65558511 0.62038577 0.61273695 0.58546724]
Number of data points in benchmark 1064
Benchmark Results
Sensitivity: 0.6071428571428571
Specificity : 0.5057915057915058
Accuracy: 0.5084586466165414
ROC 0.556467181467
TP 17 FP 512 TN 524 FN 11
None
x near_miss Data/Benchmarks/phos_PKC.csv
Loading Data
Loaded Data
Working on Data
Sample Vector [9, 9, 8, 18, 11, 9, 20, 12, 18, 6, 9, 19, 17, -2.423076923076923, 61.315384615384616, 0.07692307692307693]
Balancing Data
Balanced Data
Finished working with Data
Training Data Points: 24059
Test Data Points: 6015
Starting Training
Done training
Test Results
Sensitivity: 0.5453947368421053
Specificity : 0.6705882352941176
Accuracy: 0.6073150457190357
ROC 0.607991486068
TP 1658 FP 980 TN 1995 FN 1382
None
Cross: Validation: [ 0.5418883 0.64378324 0.62570668 0.60808114 0.59411373]
Number of data points in benchmark 1064
Benchmark Results
Sensitivity: 0.5714285714285714
Specificity : 0.48359073359073357
Accuracy: 0.4859022556390977
ROC 0.52750965251
TP 16 FP 535 TN 501 FN 12
None
In [ ]:
Content source: vzg100/Post-Translational-Modification-Prediction
Similar notebooks: